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ABSTRACT 

The 1961 meeting of the Western Regional Conference 
on Testing Problems dealt with changes in education and measurement 
since Sputnik. The following papers were presented; (1) »Who*s 
Testing Whom and for What?^ by Daniel D.^Feder; (2) "Recent 
Development and Problems in the Teaching of English** by Alfred H» 
Grommon; (3) **The New Foreign Language Teaching and the Need for 
Tests in All the Skills*' by George Scherer; (4) **The Teaching of 
Modern Mathematics** by John L*.Kelley; and (5) **some New Science 
Curricula and their Measurement** by Frederick L. .Ferris, Jr. .h list 
of conference participants concludes the report. . (KM) 
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The Tenth Annual 



WESTERN REGIONAL CONFERENCE ON TESTING PROBLEMS 

The tenth annual meeting of the Western Regional Conference on Testing Problems 
on "Changes in Education and Measurement Since Sputnik", sponsored by the Los 
Angeles office of the Educational Testing Service, convened at 9:15 a,m. in the Blos- 
som Room of the Hollywood Roosevelt Hotel, Los Angeles, California, General Cheir 
man Colonel Virgil J. O'Connor presiding. 

DR. JOHN HF MICK (Director of Educational Testing Service, Los Angeles Office): 
I am delighteu to be able to welcome all of you to the Tenth Annual Western Regional 
Conference on Testing Problems on behalf of the Los Angeles office of Educational 
Testing Service. 

This is th^ tenth ^rthese meetings and the fifth time I have had the pleasure of being 
here. Next year, if I survive, it will be six out of ten, and I will have obtained a 
majority. I realize that I am still very junior as far as some of you are concerned 
who have attended these conferences since the beginning. 

We are delisted that so many people have been faithfiU attendants over the years. 
We hope we can continue to have these conferences and that they will be of the sort 
to continue to entice you out. We appreciate very much any comments or sugges- 
tions that you may give us as to the nature, timing, or any other features about the 
conference. We wish the conference eo be useful to you, and the only way we can be 
sure is through your reactions. 

As you know, a conference of this sort doesn't just happen. A great deal of work 
goes on, and at this time I want to express my appreciation to the entire staff of the 
Los Angeles office which has been involved In one way or other. Particularly, 
thanks are due again this year to Bob Lambert, in charge of Advisory Services, for 
the overall supervision, planning, and arrangements, and to Mary Owen for the de- 
tails of seeing that everything is organized and working properly. 

At this time I am very happy to turn the meeting over to your Chairman for the day. 
He will carry on from this point. Since his name is printed on the program, I guess 
there is nothing I can do to make it a secret by saving it until the last. Our Chair- 
man today is Colonel VirgU J. O'Connor, Registrar of the United States Air Force 
Academy. Originally a Middle Westerner, he was born and reared in Waterloo, 
Iowa, earned a Bachelor's degree at Iowa State Teachers College, and won his Mas- 
ter's from the University of Michigan. He taught high school in Iowa and Michigan 
for about five years and then at the time of the war, his affiUation with the Air Force 
began. He didn't start out as a Registrar in the Air Force. In fact, he was origin- 
ally a Weather Officer. 

He received a Doctorate in Educatipn from Harvard University in 1949 and has been 
a research psychologist since that time. Apparently Harvard marks the transition 
between weather and psychology. Whether weather or human nature is less predic- 
table we will leave for him to say. 



He was a Deputy Chief in the Headquarters of the United States Air Force from 1952 
to 1955. Since 1955 when the institution began, he has been with the United States 
Air Force Academy as Registrar. Just last June he received a presidential appoint- 
ment as Permanent Registrar, one of the relatively few officials at the Air Force 
Academy who is on the permanent staff and not subject to rotation. 

I am very pleased that Colonel O'Connor has consented to be your Chairman today, 
and at this time I am very happy to turn the meeting over to him. 

CHAIRMAN O'CONNOR; Thanks, John. I wish you had started out by at least pre* 
tending to be a friend of mine. My name is Virg^, and I go by the name of Virg^U 
I am going to call everyone up here by his first name. Unless we are comfortable, 
we won't enjoy this. From now on can we do lhat, including John, and will you be 
friends with all of us up here? 

I am more than happy to be here, I assure you. Any time I can get away from Col«* 
orado this time of the year I am happy to be here, but it changes soon. Spring 
comes around the fourth of July. Winter doesn't start until the day after. Since 
there are several of us here from Colorado and I have been talking about friend*** 
ship, I can't resist telling about -* well, it's a good story and we always tell it to 
visitors. 

It is about the group from the East invited to go hunting in Colorado, and the rancher 
who invited the group wanted to be sure fhey were well taken care of* He obtained 
the services of the best guide in the Rocky Mountain region. The guide gave the 
group a detailed briefing and toward the end of his harangue he demonstrated how to 
treat snake bite. He didn't carry any snake-bite kit but employed his pocketknife. 
He said, <<Take out your knife. Now, you cut a little 'X' through the fang marks. 
Let it bleed. Suck the blood and venom out. Suck and spit, and be sure you get all 
the venom out of the arm. " He was very effective in his demonstration. At the ^d 
of the briefings he asked, "Have you any questions?" There was a question. "I can 
see how I can take care of a bite in my arm. What happ^s if I get bit in back?" 
"Mister, that's when you find out who your £riends are. " 

I do want to as^iure you that we want to be comfortable up here and have some fun 
during this day. You are going to be sitting there a long while. I hope you will be 
comfortable too. There is the old story of World War Jl of the sergeant that called 
out the recruits, "Those with ^e college degree, two paces forward, " and they 
mar ched two paces forward. "Those witix the high school diploma, one pace for- 
ward. " So he had three rows here^ witii non-graduates in the rear row. "Now, you 
college boys, you pick up the cigarette butts; you high school graduates, you pick up 
the gum wrappers; you dumb bunnies in the back row stand around and learn some- 
thing. " 

Now, I am not going to describe the theme today. It is indeed an appropriate one, . . 
"Changes in Education and Measur^ent Snce Sputnik". 

If you were fortunate to be watching television, like I was this morning, you were 
indeed thrilled. We meet on a day when we can really challenge Sputnik and celebrate 



with Lieutenant Commander Alan Shepard of the Navy. I hope some of the other 
speakers will make reference to it too. It is a great day for all of u<3 and partic«- 
ularly for those Navy boys. 

I have only one caution. I am going to be keeping time up here. We want to leave 
this room right on the dot because it has to be set up for lunch. We wish to keep 
the speakers to the precise time allowed. I remember being present at a College 
Board Colloquium where a certain speaker was more comprehensive than incisive. 
After we had listened for a while to his paper, one listener turned to the fellow 
next to me and asked, *'What follows the speaker?** My neighbor answered, *'Wed- 
nesday« «« 

The first speaker I am sure many of you know because he has traveled around the 
country a great deal as President of the American Personnel and Gmdance Asso-* 
ciation and had a hand in many other activities. Let me teU you a little bit about 
him. He was born in Philadelphia some years ago but came to Colorado sufficient-* 
ly early in life to become a graduate of East High in Denver. He went to the Uni- 
versity of Denver for his Bachelor's and continued there for his Master's. He was 
in an ai^arently accelerated program; he had the Bachelor^ s and Master's in less 
than four years after graduating from high school. Then he went to the great state 
of Iowa for the final step in his education, earning his Ph. D. from the University 
of Iowa. He stayed tixere for a time as Director of Personnel Research and then 
went to Illinois, working with high schools there on a state»wide testing program. 
He came to the University of Denver as Dean of Students and Professor of Psy- 
chology immediately after World War II. He has been at the University of Denver 
since 1946. I give you Dean Feder. 

WHO'S TESTING WHOM AND FOR WHAT 
DANIEL D. FEDER 

Thank you, Virg. Good morning, colleagues. I talked earlier this morning with 
Virg, my friend and neighbor, about this great and thrilling occasion. Subsequent- 
ly in our conversation I reminded him, too -* which was probably poor timing on 
my part concerning the size of my manuscript - that an introduction "to be immor- 
tal need not be eternal**; and Virg, in turn, looking at the paper I had with me, 
said, "Well, the same could be said for the manuscript. " I have been in the proc- 
ess ever since of trying to cut it down. 

In the long perspective of world history, xmdoubtedly there will be a period in the 
development of mankind which will be referred to as the post-Sputnik era* The 
theme of todayU Tenth Annual Conference on Testing Problems is consistent with 
the themes selected in many other phases of American life wherein an effort is 
being made to identify and appraise changes which have occurred since man's 
reach to outer space changed from dream to reality. So we too are concerned 
with **changes in education and measurement since Sputnik. One need not be a 
student of history, however, to realize that the gaining of adequate perspective 
whereby an historical trend may be traced takes considerably more than ttiree 



years. In fact, it is altogether doubtful that anything like a real trend can be claimed 
to have h j^en established in so short a time* i 

One thing is certain. Since the advent of Sputnik, the focus on education in America 
has become both sharper and more extensive than at any other time in our history. 
The introduction of bills by President Kennedy and his administration, proposing to 
distribute billions of dollars for the general support of education and for the signi- 
Hcant extension and support of the National Defense Education Act of 1958 is con- 
crete evidence of the belief that has been so often claimed but seldom supported, 
namely, that an educated populace is the most secure basis upon which to build a 
democratic society and a free world. Thomas Jefferson once wrote, **The nation 
which expects to remain ignorant and free e3q>ects what never was and never will 

We have been pecking away for many years at the fundaunental principle enunciated 
by one of the greatest of all Americans, but now in 1961 we may witness the ,first 
truly massive, significant break-through which will lead to a more adequate support 
of education and the greater possibility of insuring a totally educated and enlightened 
populace. This is surely a post-Sputnik development of significance. 

When Sputnik I went into orbit, there was a hue and cry aroused which focused on 
the failure of American sdiools and colleges to educate students in the fundamental 
sciences. As is so often the case, panic is followed by panaceas. Immediately 
after Sputnik I, a new generation of educational au^orities composed of some edu- 
eatery to be sure, but a goodly number of admirals and generals (both active, and 
retired), political aspirants, and a host of otliers suddenly became authorities. 
One of the most succinct comments on this phenomenon came iroxa Or. John Gard- 
ner, President of the Carnegie Corporation, who observed that ^^Immediately after 
Sputnik I, a great many people who heretofore had not had two consecutive thoughts 
about education suddenly went into orbit and started beeping*^ 

The burden of much of this post«*Sputnik ''beeping** was an attack upon the schools 
axui colleges for their concern witti **life adjustment'*, the "well-rounded student", 
"misiturity", and a number of other phrases whi^ I am sure you are equally com- 
petent to supply. The immediate implication of these cntlcs was that the schools 
had no business concerning thems^ves with these aspects of a student's life. For 
them the intellectual life was the only concern. 

1 am not about to argue the merits of the past or the validity of these criticisms 
in this pairticular era* I believe that history will show we have had a long and 
vivid experience of reaction, over**reaction, and compromise in these philosoph- 
ical approaches to education before Sputnik and undoubtedly will continue to have 
these adjustments in the years ahead* What I think is more pertinent for us in a 
measurement referen^ce today is the fact that so little work has been done relative 
to the "life adjustmrat" achievements of students and their measurement. Cer- 
tainly it is reasonable to suppo#e that if tibis aspect of education had been a major 
preoccupation of classroom tewbers, considerable attention might have been de- 
voted to the development of vaU3 and significant measuring instruments which 
would appraise the student* s growth and development in these dimensions. The 



fact that after all these years one still has great difficulty in finding really signifi- 
cant instruments for measurement in this area as compared with the quality of other 
measuring instruments must certainly be interpreted as giving the lie to the claim 
that such adjustments have preoccupied the teacher beyond the essential commit- 
ment to subject matter qua subject matter. In fact there are certainly a great many 
of us here who would agree, I am sure, that the classroom teacher serves his best 
function in developing minds in relation to subject matter and may very well be 
treading dangerously when he branches out into the personal and life adjustments of 
the student, 

I too would wish to express concerns about certain test developments, and mine are 
quite specifically related to the post-Sputnik era and some of the things that have 
happened in all levels of education as many test users seemed to go overboard for 
testing. There are some perspectives necessary for the proper evaluation of these 
concerns. Although significant programs for the identi&cation of talent were under 
way before Sputnik, many of them did not get their full impetus urtil after we pan- 
icked, Wolfle's highly significant evaluation of our talent resources was conceived 
long before the space era.^** Project Talent** was on the drawing boards and inits 
planning stage before either we or the Russians blasted off. In 1958, the Rocke- 
feller Brothers Foundation report, **The Pursuit of Excellence", laid before us a 
host of challenges^ one of the most importamt of which was e:»q>ressed in these words: 
''A wasted skill, a misappplied ability is a threat to the capacity of a free people to 
survive". In this sentence alone one might find almost all the justification needed 
for the National Defense Education Act and the tremendous impetus it has given to 
the effort to identify talent at every level. But there is a corollary statement in 
this document also which needs to be borne in mind whenever we turn our attention 
to the identification of the talented and able student, and that too is a basic principle 
on which our civilization and democracy have been founded* It was expressed in 
these words: "Judgments of differences in talent are not judgme: 3 of differences 
in human worth". I propose that these two statements with their tremendous im- 
plications for education form a charter for the ensuing comments so that we do not 
lose sight of the individual student who still is and always was the focus of our edu- 
cational efforts. ' 

Despite criticism of minutiae, advances in the testing of aptitude and achievement 
have been well groimded. No longer are we confined in educational testing to the 
reproductive memory challenge which characterized TV shows of a few years ago 
and which, as carefully noted by the announcer, is still the major characteristic of 
the well known College Bowl program. We can successfully and demonstrably 
measure the ability to reason with given facts, the ability to make inferences from 
data, and the ability to integrate knowledge from several sources in problem- 
solving situations. To be sure, none of us would argue that these alone are the 
full measure of an individual's achievement or future learning capacity. We have 
even begun to probe into the more meaningful aspects of personality and made 
efforts to evaluate the total personality as a fxmctioning entity. 

Despite these advances there are still many people outside our professional ranks 
opposed to the use of any kind of tests and measurements, largely either because 
they donH understand them or because in some way personal toes have been stepped 
upon in the course of a testing program. For example, in a large city school 
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system about three years ago a virulent attack wa^ laxmched against the entire pro- 
gram of testing aptitude and achievement, spearheaded by one mother whose son 
had been found seriously deficient in both achievement and aptitude and was accord- 
ingly placed in sections for retarded children. The affront was sufficient to stimu- 
late the attack upon every effort the school system was making in the direction of 
better adaptation of its methods to individual student needs. 

V/itness also the diatribe ageiinst the College Entrance Examination Board which 
appeared in an issue of Commentary last year. The Board, in reply, presented 
the data on a study in which a group of high level adnaissions officers and college 
deans made predictions about students' achievements in college, using all of the 
non-objective data that they normally had at their disposal, and the results were 
pretty miserable as compared with the predictions obtained by the use of some of 
the College Board examinations. 

It seems safe to predict that the tremendous impetus given to testing programs for 
the evaluation of individual students during the past three years under the influence 
of the Natioxal Defense Education Act of 1958, itself unquestionably spurred by re- 
action to Sputnik, will be reflected in a widespread common acceptance pf^his^type^ 
of testing program as an essential part of any school's operation. What is perhaps 
not so clear yet in this area are the inherent dangers and naivete which accompany 
the tmcritically enthusiastic tendency of us Americans to hop onto any attractive 
bandwagon and go along for the ride without te^nwing or attempting to foresee the 
end goal. I would not for a moment suggest that we should neglect the technical 
imperfections of some of the tests offered on the market daily. But in actuality 
we ought to be more concerned v^th the greater and more significant imperfections 
in the test users rather than with imperfections in the tests. 

Under the sponsorship of the American Association for the Advancement of Science, 
a group of leading test technicians met in Ann Arbor, Michigan, almost exactly 
three years ago. With the impetus of the N* D. E. A. of 1958 already behind them 
and the rapidly developing national and state scholarship programs on all sides, the 
committee addressed itself to the role of tests in state- wide counseling and scholar- 
ship award programs. The committee enunciated three general principles to gov- 
ern the choice and scheduling of aptitude measures: 

**1. The choice, timing, and use of aptitude measures should be planned so 

as to supplement as effectively as possible the information provided by 

measures of achievement. 
2. Information provided by measures of achievement should be reasonably 

current at the time of its most critical use. 
3« Any school-wide testing program should be supplemented by additional 

individual testing as determined by the special problems presented by 

the individual case. 

Note in these principles the recognition of the fact that the achievement testing 
program is inherently more significant to the school system and hence more basic 
in the service of its needs than other testing programs. Beyond this was the re- 
cognition of the principle mentioned earlier that an effective measure of achieve- 
ment is still one of our best predictors of future performance. 

ERLC 
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The committee went on to recommend that a scholastic aptitude test should be given 
at the sixth or seventh grade level, designed to yield scores based on verbal and 
quantitative materials.. It next recommended a test of essentially the same sort to 
be given at the eighth or ninth grade level approximately two years later. Its third 
recommendation was for a comprehensive test of academic ability to be given in 
the tenth or eleventh grade. This test was seen as one that could be useful in iden- 
tifying students eligible for later scholarship competitions, to make certain that all 
students with college level ability would be encouraged to plan for college and under- 
take study programs to meet college entrance requirements, and also to be used by 
counselors in assisting the student in choice of college or other post-high school 
education. A fourth recommendation included a vocationally oriented aptitude bat- 
tery for the eleventh or twelfth grade, particularly for those who were not plaiming 
to go to college. 

The foregoing recommendations of a minimum essentials program took account of 
the school^ s needs for the evaluation of achievement as well as for the individual's 
needs for itxformation upon which to make his important life choices. Lest the 
orientation and attitude of this committee of test experts be misunderstood, let me 
hasten to add that they recommended five categories for data collection and a sched- 
ule for each beyond the objective testing program just outlined. These included a 
biographical data folder, interests as measured by tests and other devii es, per- 
sonality testing, parental interviews, and teacher appraisals* The importance of 
these latter data must not be overlooked, although our focus today is on the field of 
testing. 

It would be interesting indeed to attempt a thorough-going census of school practices 
around the country in the light of the foregoing recommendations. Through associ- 
ation with a number of scholarship programs as well as with the admissions opera- 
tions of my own University, I have the opportunity to see literally thousands of stu- 
dent reports each year. In many school systems the conventional transcript is no 
longer adequate for the reporting of test scores. Instead, one-, two-, or three-page 
mimeographed profile sheets accompany a transcript of grades. In numerous in- 
stances these reports reflect the fact that multiple testing of the same characteristics 
is being done either on the same day or a few weeks apart. Oftentimes the profile 
will reflect only achievement tests or only aptitude tests. V/hether or not both sets 
of data may be available at the school itself is not evident from what is provided on 
the profile. To the counselor, the admissions officer, or anyone else concerned 
with the identification of operational talent, a series of scores at the eleventh grade 
median for an eleventh grade student on the Sequential Tests of Educational Progress 
will have one set of meanings if the student's academic potential as measured by an 
equally valid test, of aptitude also places him at the medium of his class. But if the 
potential is in the top ten per cert and the achievement is at the median, the scores 
indeed have quite a different meaning in reflecting the motivation, application, and 
earned increment which the student is carrying away from his educational experience. 
The basic principle to be observed here is that the testing program must be so con- 
structed, not only to differentiate one student from another, but also to make possible 
differential appraisals within the student himself. Obviously the challenge to both 
counselor and teacher rests in the stimulation of the student to the optimum achieve- 
ment of which he is capable as demonstrated by valid measurement of his potentials. 
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Teat makers have dc/ised a varied and interesting series of methods for the report- 
ing of test scores. In the three years plus since Sputnik, little has been done to 
eradicate the IQ from common parlance in testing situations despite the emotional 
anii unrealistic thinking which has char act eri'^ed this concept. Efforts to substi- 
tute percentile scores and standard scores have made inroadis but have been re- 
sisted because they are both more mysterious and more difficult to comprehend by 
lay persons. A notable advance in test realism was made with the introduction of < 
the concept of the percentile band when the School and College Ability Tests were 
placed on the market* This was a frank, honest and realistic recognition by the 
test makers of the concept of variability of a test score and the necessity of inter- 
preting it within a band range which would encompass probabilities of any given 
score being tiie **true" score for an individual student. Here the test makers have 
made the effort to guide the test users to enlightened and accurate methods of in** 
terpretation but the plain fact is that little is known yet about the best way in which 
to report test scores* 

One is tempted to raise the question as to whether, for counseling and educational 
evaluative purposes, we actually need more than four or five categories into which 
to classify students by means of test scores instead of focusing vith so much assur- 
ance and exactitude upon numbers whose absolute reliability is not in any way as- 
sured except as they may be the result of repeating testings over a number of years. 
Obviously, for research and statistical purposes^ numbers still are necessary. But 
it is of questionable validity to make discrimination between tv^'o students on the 
basis of a twenty-five point difference on a College Etetrance Examination Board 
test where the score difference may be the result of a performance difference of 
only three or four items. If one wishes to push this analysis still further and ques- 
tion how the classroom teacher, using generally even cruder instruments than 
those we are concerned with here, can justify a numerical grade for one student 
of 92 2uid for another of 93 and prov*» that there is an actual difference, perhaps 
all I can say in reply is, **How naive can we be?" 

Although I have actually tried to encapsulate a number of specific points and deal 
with them as entities, there is in fact an essential inter- correlation among all of 
these matters under discussion. There is no gainsaying the fact that there is a- 
vailable today more information of an objective and reliable nature concerziing 
the aptitudes and achievements of students at all levels of education than has ever 
before existed in our history. In addition to internal school testing programs, 
thelre are now state-wide and national testing programs, ail returning results to 
the schools and all having certain common motivation - the identification and stim- 
ulation of students of all levels of ability to the maximum achievement of which 
they are capable* This proliferation of programs and multiplication of available 
data is not the child of Sputnik though it be essentially a post* Sputnik phenomenon. 
A number of state and national programs existed long before. 

Certainly the national testing programs sponsored by the College Entrance Ex«- 
amination Board, the Educational Testing Service, the Psychological Corporation 
and other similar agencies were under way many years before the realities of 
space travel* But their essential purposes were much the same at inception as 
they are now. 
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I The challenge to identify and stimulate able students raises the interesting question, 

* «»By what means?*' If the student is never apprised of his potential, then to what 

extent can he be motivated to use it? If the information lies fallow in the files of 
the school or in the counselor's office, who is testing whom and for what purpose? 

J The time for secrecy is long since past. It should be needless to point out that at 

1 all levels of education the schools and school staffs, the students, the parents, and 

all concerned are in a most intimate partnership which can operate effectively only 
as the partners treat each other with condor and integrity. On this point the testing 
j leaders at Ann Arbor said, "The forthright interpretation of test results, properly 

related to other available data, should be made to the pupil, his parents, and his 
I ^ teachers. Care should be exercised, however, tb' t data are not over- 

I emphasized. It is important that results be inter i u^^a not merely reported. " 

r If there is a purpose in giving a tost of any kind, if there is a purpose in collecting 

I data of any kind, then certainly the most essential purpose is that of sharing the 

information gained thereby with those who have the greatest concern, and in this 
j I include students, parents, teachers, and others who may rightfully have reason 

I to use the information in the student's behalf. To the objections and fears that 

test data may be misused by persons who are not fully sophisticated in the inter- 
j pretation of their meanings, the only answer is that it is the responsibility of those 

I of us who introduce these lests into the school program to make sure that those 

who have a right to the data are educated in their proper interpretation. In truth, 
J it might be more valuable as part of the teacher preparation program to expose 

the potential future young teacher to a course in the appropriate use and interpre- 
tation of tests and measurements in place of part of the busy work which charac- 
I terizes some of the courses now required for teacher certificates. Education has 

i moved forward dramatically and drastically in the last few years in both philosophy 

and practice, but teacher certification requirements have not kept pace. 

j 

' the past few months a school system in this general geographic area came 
under fire for certain of its testing practices. Pertinent to our present discussion 
I was the discovery by an investigating committee that an elaborate testing program 

1 had been undertaken without any preconceived notion of what the program was de- 

signed to do or what its ultimate objectives would be. One wonders to what extent 
this kind of rather random testing has become prevalent through the availability of 
i N, D* E. A. funds. Testing for the sake of testing alone is as naive and fruitless as 

^ the ignorance-based or prejudice-based total rejection of tests. Delineation of the 

^ I objectives of the testing program is a first requisite. Only in such terms can in- 

telligent selection of tests be made. The culminating goal of the program must be 
the interpretation of the results to those who have a real stnd vested interest and 
are entitled to such information, because it is only in terms of such practices that 
the evaluative, predictive, and stimulative objectives of the testing program can be 
accomplished. Continuing, meaningful reports of results to the supportive com- 
munity are an inherent part of this plaiming. 

_ In recent months a number of professional associations have concerned themselves 
I with the multiplicity of national testing programs which seem to have common ob- 

jectives. An effort lo evaluate school reactions to the inherent problems was made 
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recently. It is altogether intriguing to note that two separate investigators, both 
reputable and competent, came up with quite opposite conclusions from their anal- 
yses of common data. Actually, the national testing program does not require re- 
leased time from ordinary class duties* The internal, local testing program, on 
the other hand, does take the students out of normal classroom activity and in some 
cases for many hours in the course of a school year. Oftentimes the two programs 
provide very much the same kind of data concerning the student, with no great en- 
har>r^ -ien<- of the overall picture being provided by the duplication. Having literally 
CO J le nation during the past year in a series of meetings with counseling 
groups III many different states and areas, I feel that, far from being the imposi- 
tion which some have claimed, the national program does less to interfere v/ith 
school procedures than do many of the internal local programs* What is far more 
discouraging in this respect is the fact that altogether too few of our schools and 
colleges are using the data available to them either for understanding the student 
himself or for helping the student to a better self-understanding; and lack of staff 
in adequate numbers is not the only conditioning factor. 

Although I cannot document this next observation with a survey of facts and numbers, 
I am nevertheless quite convinced that the old threat which national and standard- 
ized testing programs used to pose as a restrictive influence on curriculum develop- 
ment has pretty well disappeared. The impetus which has been given to curriculum 
development through the work of the College Entrance Examination Board and the 
Educational Testing Service, as a case in point, and the correlated developments 
in improved exctmination methods have indeed operated as a stimulus to classroom 
teachers to re- evaluate and reconstruct curricula in many subject-matter areas. 

As recently as a decade ago one heard many concerns expressed about the possible 
restrictive influences that would be imposed upon teachers when their students 
would be tested by external, objective measurements. To be sure, there are some 
areas in which these concerns are still voiced because programs have been narrowly 
conceived* But on the whole, with the tremendous improvement in measurement 
methods which has been achieved in the past few years, many teachers find them- 
selves challenged to a higher level of instruction and have more significant guides 
to content than they have had before as they strive to meet the standards set by 
well- conceived and well-constructed examinations. In truth, many of our best 
standardized examinations are extending the frontiers of curriculum and teaching 
methods and posing challenges rather than imposing restrictions* There is evi- 
dence to suggest that we are doing a better job of measuring the student^ s ability 
to integrate, apply, and make inferences from his learning than we are in the actual 
teaching of the processes involved* 

These remarks opened with the notation that this is a conference on testing prob - 
lems . The essential focus to which we are to address ourselves is in the realm of 
developments resulting from the impact of, or at least since the impact of. Sput- 
nik. Judging from the titles of the talks which are to follow, we will be treated to 
reviews of recent expei^iments and developments in teaching and testing in English, 
mathematics, foreign languages, and the sciences. Even the casual student of de- 
velopments in these areas must know that totally new concepts of subject matter 
depth and quality are being probed and developed at several levels of education* 



From my perspective there are problems, not only in the technical development of 
tests themselves, but even more significantly in the uses to which the tests are put 
and the persons who are using them. I see these as continuing challenges, not 
merely as problems for which some ready solutions will be found. Certainly there 
is no reason to expect, in a field as dynamic as education is or ought to be, that 
static solutions are a possibility. Rather it is more likely that new solutions with- 
in themselves will create or expose additional problems. 

Although we have maximized the predictive efficiency of many of our t6st instru- 
ments as used in the present structure of educational evaluation, the obtained cor- 
relations leave much to be desired for individual predictive efficiency. Obviously 
there are still unmeasured elements in the individual learner as well as in the 
learning process which require much further study. I hesitate to predict how long 
we shall be beset with IQ's of doubtful meeuiing and little comparability. We need 
to give attention to new and more meaningful ways of reporting test scores so that 
they may be more readily interpreted and tuiderstood by counselors, teachers, 
students, and parents alike. The need for such an experimental study becomes all 
the more urgent as we face the prospect of tremendous increases in enrollments 
at all levels of education and an equally tremendous shortage of qualified person- 
nel to provide the individualized kind of interpretative service which is an essen- 
tial part of Euiy testing program. As we look more and more to possible mechan- 
ical devices and means whereby students may intelligently interpret their own 
scores and come to better understanding of themselves, it will become increasing- 
ly important to have available simple, standard, but universally meaningful inter- 
pretation methods. Standardization of terminology is a characteristic of maturity 
in any science. In the science of measurement we need to grow up in this regard. 
Using tests wisely and well is a prime requirement in program planning and ulti- 
mate program culmination. 

All too often the student is seen as a profile of tests translated into some kind of 
scores which enable us to compare one aspect of his development with another but 
oftentimes each without reference to the totality of his person. The challenge to 
the test user is that of seeing the student as a total human .being and not as a 
test profile. The challenge is to appraise the student as a functioning entity in 
which the test profile is merely one device for contributing to his understanding 
and appraisal of himself rather than the total global description of the student as 
a vital human being. Although biographical data, anecdotal records, interest, and 
personality descriptions are not the central subject of today's discussion, never- 
theless they are part and parcel of the testing problem. 

Test users must become knowledgeable in how to relate test-derived information 
to other student data which permit the description of the total personality and are 
integral to our understanding and appraisal of the student. In addition, it is im- 
portant that we do not oversell the advantages that accrue from testing. It is dis- 
tinctly more dangerous to oversell the value of tests than it is to undersell them 

The support of testing programs under the National Defense Education Act must 
be used wisely and with careful interpretation to the community elements who have 
made the effort possible. The indiscriminate use of tests, the failure to plan 
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programs carefully in terms of ultimate use, and the often implied assumption that 
the tests are ends in themselves may produce expectancies which we cannot fulfill? 
In the light of such failures, we may witness the loss of all the good we would accom- 
plish by virtue of our inability to compensate for some of the evils unwittingly intro- 
duced to the programs* 

Fully as much as Sputnik and Explorer, and now Mercury, a lot of us have gone into 
orbit in the past three years and have become test-happy rather than test- wise. We 
search every new test as though it may have in it the panacea for every educational 
ill. Yet the basic fact remains that we are far from having exhausted the potential 
information available to us in many of our existing tests if we but know how to read 
all of the signs of the student's performance on them. 

I do not know the name of the sage who observed that, »»Nothing seems as long to man 
as life when he is beginning it nor nothing so short as life when he is ending it* ^* I 
find myself precisely in this position at this moment. At the outset it seemed as 
though there would be far more time than needed to reflect upon the simple questions 
of who is testing nowadays, whom we are testing, and what our purposes in testing 
are. Suddenly I find myself with so much more that ought to be said and hoped for 
against the inexorable march of time. If I have read the trends right, the evaluation 
of human abilities and achievements will in the next decade become a central focus 
of all concerned in the preservation and utilization of what is our most precious nat- 
ural resource-*-an educated and enlightened citizenry. 

CHAIRMAN O'CONNOR: Very fine, Dan. That was great. In addition, thank you 
for staying within the time limit. It does give us a chance for some questions. 

Now this is your opportunity to get into the conference. Every questioner will have 
his question recorded. Because of that, I want to be sure that when you do get up 
to ask a question, you give your name, position, and place so that we can record it. 
Do we have a question for Dean Feder? 

MR. JOHN W. McDANIEL (Vice President, Instruction, San Bernardino Valley Col- 
lege, San Bernardino, California): I am concerned with one of the very fine early 
statements that Dean Feder made, Judgments of differences in talent are not judg- 
ments of differences in human worth* Do you think we have made any progress in 
this dimension? 

DEAN FEDER: I wish I could take credit for the uniqueness of the words. I cannot. 
I can only say I have shared the thought many times. These words, as you recall, 
were a quotation from the Rockefeller Report. My insistence and continuing repeti- 
tion of that statement — and I will repeat it whenever I am given a chance-- stems 
from the fact that there has been a very serious tendency to misinterpret some of 
the intent of the language of the National Defense Education Act and also to misin- 
terpret the entire trend toward the measurement of talent or the measurement of 
ability. The great danger which I perceive here is that in our reaction and over- 
reaction tendencies, we may find ourselves certainly trying to stimulate the sup- 
erior student, something which we have needed to do and have really just begun to 
do in the last few years; but we are also in the process then of relegating the average 



or less-than -average student to some other much less acceptable status in life. 
This I believe would be a violation of the Jeffersonian principle of man^s equal- 
ity in the eyes of God. 

One could go on with this. The whole emphasis of the comments of Rickover and 
even some of the implications of Conant's recommendations seems to me to point 
in this same direction. If the student is not talented, if he is not able to move on 
to college and through college, perhaps on into graduate work, then we relegate 
him to something less worthy, and essentially to a sort of vocational peonage'. This 
I think is a dangerous element and I defy it. That is also why I think we need to 
keep the words ever present, ever foremost in our kind of society, *'The judgments 
of differences in talent are not judgments in differences of human worth, " 

MR. SAMUEL KAVRUCK (Specialist, Testing Programs, Guidance, Counseling, 
and Testing Section, U. S. Department of Health, Education, and Welfare, Office 
of Education, Washington 6, D. C. ): Now, I'd like to mention to Dean Feder that 
this problem of how best to interpret the test scores has been a great concern, 
particularly one of our specialties, Dr, McLaughlin of People Appraisal Section 
has recently tried to make it known that we would like more information about this 
important problem; and I think can all agree that the answer will come, not 
from the test makers, but from the people like those here now, the people who 
are to actually interpret the scores, not the parents and pupils, I wonder if I 
might ask each of you here to inform us in the event he has developed methods of 
interpreting which he feels are perhaps new and productive. We will try to collect 
them and feed them back to everyone. Thank you. 

CHAIRMAN O* CONNOR: I feel that we should proceed. If you have additional 
questions for Dean Feder, you may ask them later. 

I want to tell you about a pre^Sputnik experiment in Iowa, Dean Feder and others 
will remember Professor Seashore who was one oi the originators of aptitude test- 
ing, particularly musical aptitude. I recall vividly an experiment in the town of 
Leando in Southern Iowa. There is a little town of Douds on one side of the Des 
Moines River and opposite it, Leando. These two towns were always competing 
in various ways, particularly where their brass bands were concerned. Well, 
finally Leando got a tuba, the biggest one seen up to that time, A tuba for their 
brass band so as to outdo Douds. Then, they had the job of finding someone to 
play the tuba, and having heard of Dr. Seashore and his testing experiments they 
decided they would measure aptitude for tuba playing. 

It was a good experiment. Many tried and many failed the aptitude test; and they 
eventually got around to the blacksmith who wished to have his aptitude for tuba 
playing measured. They had only three items in the test but it was a fairly valid 
test- -fair face validity, at least. The first question put was very simple, "Can 
you carry a tune?" The blacksmith whistled "Dixie" to prove his adequacy. 

He then was asked, "Do you have the strength for tuba playing?" He picked up 
the anvil by the horn and swung it over his head. 
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Third, ''Do you have the wind for it?^* And the poor blacksmith didn^t know how to 
deinon8trate» until just then the Minister canne trotting up on a little old gray mare^ 
As he was tying her up to the blacksmith's shop, the blacksmith's eyes brightened 
and he was able to demonstrate his tuba aptitude. He walked over to that little old 
gray mare, lifted up her tail and blew the bit right out of her mouth. 

Now, we have heard the only presentation with the word "testing" in it. I am sure 
all of the following speakers will use the word, too; but, as a preliminary to the ones 
to follow^ I would like to call your attention to an item, a very recent item, in the 
newspaper. Perhaps some of you have read it but I want to be sure all of you know 
of it. It was in regard to an "Opinion Poll" by the Institute of Student Opinion. There 
was the question: "Should the American high school place more, less, or the same, 
emphasis on the following subjects: agriculture, commercial courses, English, for- 
eign languages, home economics, mathematics, music and visual arts, physical ed- 
ucation, science, shop, or social studies?" The results were interesting. We donH 
always realize how much these yoimgsters are willing to do or what they really want 
to do. 

Here are the results for the boys who answered the "Opinion Poll": 75 per cent 
wanted more emphasis on science; 73 per cent, on mathematics; 62 per cent, on for- 
eign languages; and 51 per cent— more than half of the boys- -wanted more emphasis 
on English. 

Here are the results for the girls: 68 per cent wanted more emphasis on foreign 
languages; 65 per cent, on science; 61 per cent, on English; 60 per cent*-almost 
the same- -wanted more emphasis on mathematics. 

I could go down the whole list, but those were the ones emphasized most by the stu- 
dents. Of course, there were some that wanted less* Twenty- six per cent of the 
boys wanted less emphasis on music and the visual arts. Of the girls, 19 per cent 
wanted less emphasis on shop. That's fetir enough. The percentages for more em- 
phasis do illustrate rather remarkably that our youngsters want a lot more than 
they are getting in what we usually call the tough subject-matter areas* 

The next presentation this morning is by one of your neighbors from the North. 
However, Professor Alfred Grommon came fcom the East as so many of us have. 
His graduate work was done at Cornell and Hiat institution granted him a Ph. D* in 
English. Professor Grommon, like so many of us here at the table, has b* ^ 
high school teacher. He has been at Stanford since 1945 and for six yeai*^ Di* 
rector of the Freshman English courses. For some time he has been Associate 
Director of the Commission on the English Curricultim of the National Council of 
Teachers of English. At the University he is an Associate Professor of Education 
and English. Al Grommon. 

RECENT DEVELOPMENTS AND PROBLEMS IN THE TEACHING OF ENGLISH 

ALFRED H. GROMMON 
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Thank you, Virg. I do have a few bits to utter but I hope to do it all by myself. 
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I Howeveri if I prove unable to do so I will seek assistance from you because I am 

sure that you have been, not only a registrar, a high school teacher, and a mete* 
( orologist, but a blacksmith as well* Being associated here with the gentleman from 

I Colorado reminds me of what I saw on campus the other day« Some student with a 

car from Colorado had a sign on the back, much like signs on cars from Texas, 
f "Made in Texas by Texans. *^ However, this fellow from Colorado, probably in re- 

I bellion against such things, had a sign which stated, ''This car wasn't made by 

anybody, *^ 

I 
1 

I I should like to talk for a few minutes about some recent developments and problems 

in the teaching of English and, as Dean Feder mentioned, this morning's news col- 

j ors my opening comments, too; but what I have written is this: 

I 

Russia certainly deserves credit for her scientific and technological achievements 
in putting the first satellite in orbit on October 4, 1957, and for putting Yuri on his 
cosmic merry- go- roimd in early April. But we caimot let them claim credit also 
for such achievements as inventing the game of basketball and for what is now hap- 
I pening in the teaching of English in our elementary and secondary schools. To be 

sure, shock waves of that October orbiting a^ccted, at least temporarily, teachers 
and pupils in fields other than science and mathematics. Yet to suggest, even by a 
i handy cliche, that sigxoificant educational developments may have been latmched 

^ here in the wake of Sputnik would be a disservice to the many people responsible 

for important programs and developments begim decades ago. 

I 

But probably red-faced America has indirectly colored somewhat the present teach- 
ing of English. Parents, school board members, administrators, college profes- 

j sors, legislators, newspaper editors and columnists, and English teachers them- 

selves are now taking a sterner look at what is going on in the public schools. Many 
of these people, however, seem motivated also by several other forces more notice- 

I ably affecting their families, their children, and their teachers right now. 

Yet in my many visits to English classes in* jimior and senior high schools, in my 
I working with many student teachers and, as a curriculum consultant, with depart- 

ments of English and school districts, I see no evidence of any revolutionary change 
^ in many pupils* attitudes toward the importance of I^glish. The significant excep- 

} tion, of course, is the encouraging attitudes among the best students. What I have 

to say about developments in the field of English suggests that teachers are busy 
striving to improve their programs and performances, but I am not yet prepared 
j to grant that any comparable deep commitment to the importance of English exists 

among the majority of students in secondary schools. 

' I should like to comment briefly upon developments in curricula for English classes, 

in co-ordinating programs of the elementary and secondary schools with those of 
the colleges, and in the teaching of granunar, usage, composition, and literature. 
I am not implying, however, that tiiese examples represent the \riews and efforts 
of thousands of teachers of English who seem either unaware of these trends or in- 
different to them. Nevertheless, in many commtanties, large and small, encour- 

I aging work is going on. 
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As to the first development: for at least the past quarter of a cer.tury, departments 
of English in junior and senior high schools have been trying to create sensible cur- 
ricula and curriculum guides based upon research and published reports of effective 
practices. These publications have helped teachers take into account research on the 
characteristics and needs of adolescents, to relate English to their lives, accommo- 
date the wide range of individual differences, develop thematic units for the teaching 
of literature, and use a variety of materials and activities suited to the diverse abil- 
ities, interests, backgrounds, and purposes of pupils who are required to study 
English throughout the junior and senior high schools. They have also urged teachers 
to co-ordinate English programs throughout the elementary and secondary schools and 
with those in the colleges. 

The second, a more recent plan for co-ordinating programs in the schools with those 
in the colleges and for grouping pupUs on the basis of special abiUties and interests, 
is the Advanced Placement Program. This important development grew out of the 
recommendations in the report, General Education in School and College , prepared 
by representatives of the faculties of Andover, Exeter, Lawrenceyille, Harvard, 
Princeton, and Yale, and published by the Harvard University Press in 1952. This 
group sought some way of co-ordinating programs in secondary schools and colleges 
so that able students might not only avoid repeating in college what they had already 
studied in school but also get college credit and/or advanced placement for superior 
preparation. The group's recommendations were first applied and developed by The 
School and College Study of Admission with Advanced Standing, whose program of 
classes and national examinations began on a modest scale in 1953. Shortly tiiere- 
after, the direction of the plan was assumed by the College Entrance Examination 
Board, whose officers had already helped the School and College Study group initi - 
ate its plan of admission with advanced standing. This plan has significantly 
strengthened EngUsh programs in a rapidly increasing number of public and inde- 
pendent high schools throughout the country, has improved co-ordination of these 
English courses with those in colleges and universities, and has helped improve 
the quality of teaching not only in Advanced Placement English classes, but also in 
other related English classes. 

In 1953 only seven pilot public and independent schools began offering advanced 
^-irses. Eleven other schools agreed to present candidates for examinations in 
fv*ay, 1954. Twelve colleges offered to consider for advanced standing these can- 
didates prepared in accordance with the recommendations for the program. In 
1953-5 4 these 18 schools presented 532 candidates who took 959 examinations and 
enrolled in 94 colleges that fall. In 1959, 890 schools presented 10, 531 students 
who took 14, 158 examinations and later enrolled in 567 colleges and universities. 
Examinations are available in eleven subjects commonly taught in schools and in 
the first year of college. Each year, however, about 40 per cent of the examin- 
ations are in English. 

This program represents a major effort to improve the quality of English courses 
and instruction in secondary schools. But I think it is also an effective means of 
stimulating capable students to make tiie most of their academic abilities and of 
advancing their high school and college studies. By its pervasive effect upon the 
intellectual atmosphere of the school, it is one of the most potent weapons for 



17 

combating many high- school students' educational apathy and even outright resist- 
ance to learning. Even though the pcirticipation has increased dramatically from 
18 schools in 1953 to 890 in 1959, the total is still relatively insignificant when 
compared to the potential in the 26, 000 secondary schools in the coimtry. 

The third development, a somewhat related plan designed to improve the prep- 
aration of students for college English, is that of the English Commission of the 
College Board. This Commission will try to improve students' preparation for 
college English by retraining selected teachers of secondary- school teachers by 
means of summer institutes on twenty campuses beginning in 1962 and by publish- 
ing syllabi suggested for classes designed for college preparatory students. The 
effects will not begin to be noticeable, however, imtil sometime after the 1962- 
63 school year. 

A fourth factor affecting English progrcims in California public secondary schools 
is the voluntary system of accreditation for high schools ertabHshed by the Cali- 
fornia Association of Secondary School Principals. To qualify for this service, 
a district must have written courses of study. Because masiy schools in Califor- 
nia do not now have courses of study or curriculum guides, the English teachers 
in many districts presently are constructing courses and guides. In some schools, 
these new guides, including courses and materials for Advanced Placement Eng- 
lish, are noticeably affecting the quality of instruction. 

A Hfth development is the increasing use of flexible schedtiling of classes, and a 
sixth is the growing use of team teaching among teachers of English. 

The final development to be mentioned here is the report on the status of the 
profession of teaching English, Th e National Interest and the Teaching of English, 
published by the National Council of Teachers of English. This report was pre-* 
pared to inform the nation and particularly the Congress of the United States on 
the status of the profession and on the importance of national support for the prep- 
aration of teachers. It has been enthusiastically received by the press. «In time 
it will affect the preparation of teachers of English and the quality of the English 
programs and teachmg in the schools. One grave national problem is the inade- 
quate supply of teachers prepared to teach English. According to this report, 
only 40 to 60 per cent of the teachers of high- school English have completed col- 
lege majors in English. Let me focus this statistic on California. I don*t have 
recent figures on this problem. But the Principsds^ October Report of 1955 was 
on the teaching of English in the junior and senior high schools in California. The 
figures reported at that time on the preparation of teachers assigned English . 
classes indicate that in 1955 approximately 250, 000 students in Califomia public 
junior and senior high schools were taught English by teachers who had neither a 
major nor a minor in college English. 

Other forces are also having encouraging effects. The increasing percentages of 
high-school graduates entering college have put pressure upon some pupils en- 
rolled in English classes. Some teachers are assigning more papers to be writ- 
ten and more material to be read. Probably all are trying to find time to read 
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pupils' compositions more carefully* Educational Testing Service's pioneering 
development of lay-reader programs has helped many school districts throughout 
the country provide English teachers with competent help in evaluating students' 
writing. Reports on college students' reading habits have awakened some teach* 
ers to the importance of their responsibility of helping students become lifetime 
readers of books« 

School boards' willingness to face facts and to take action » expensive action - is 
an important factor affecting the teaching of English. For example, in 1958, the 
New York City Board of Education decreed that no pupil retarded two or more 
years in reading ability was to be promoted. Automatic promotion was abolished. 
The delayed consequence of that action is now becoming painfully apparent. On 
April 14, 1961, Dr, John J, Theobald, Superintendent of Schools, reported to the 
Board of Estimate when he presented his budget for the city's schools for next 
year, saying that **67, 067 of the city's 172, 000 junior high school pupils were more 
than two years behind in reading, • • 10, 000 seventh-graders cannot read third*-grade 
books, • • 15, 000 elementary school pupils were in danger of not being promoted this 
term because of reading difEculties. " Then Dr. Theobald requested $3, 000, 000 to 
liire 560 remedial reading teachers. He didn't say where he is going to find these 
teachers* But he did state that "the cost of having 15, 000 elementary pupils re* 
peat a year of schooling would come to nearly $8, 000, 000. 

Here is indeed a courageous confrontation illustrating school boards' and admin- 
istrators' toughening frame of mind toward public education, a frame of mind that 
presumably is also strengthening some English teachers' resolve to hold pupils to 
higher standards of performance. The evidence emerging from New York City's 
expanding program called Higher Horizons and that reported by Martin Mayer in 
his article, *'The Good Slum Schools, published in Harper's Magazine, April, 
1961, challenge the validity of nay earlier statement about many students' indif- 
ferent attitude toward the learning schools offer. I hoped to be proved wrong. 

But what are some of the specific recent developments in various aspects of the 
teaching of secondary- school English? One major development, especially since 
World War II, is the teaching of descriptive modern English granunar to supple- 
ment valid aspects of conventional grammar or, in some cases, to replace it 
entirely. Linguistic analysis of the structure of language has a long history. But 
English teachers in our secondary schools became aware of it after the appearance 
of Charles Fries' The Structure of English in 1952 and Paul Rober ts' Patterns of 
English in 1956. So far, Roberts' book is the only high-school text based entirely 
upon structural linguistics applicable to secondary- school English classes. Many 
articles on the nature of descriptive grammar and teachers' use of it appear reg- 
ularly in The En glish Journal and College Enpilish, journals that are read by about 
50, 000 teachers of English throughout the cotintry. Yet according to Harold Allen's 
recent survey reported in the National Cotmcil's report. The National Interest an d 
the Teaching of English, only about 17 per cent of the institutions preparing teach- 
ers of English require candidates to study modem English grammar. Teachers of 
English in secondary schools and colleges who presumably have scholarly attitudes 
toward other aspects of English resist accepting the ever^^increasing knowledge 
^ about our language reported by linguisticians and structural grammarians. As 
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Nelson Francis has said, **Here is a cultural lag of major proportion* But 
the number of teachers using these findings is gradually increasing, and more 
textbooks are being prepared* 

Even though these usable aspects of conventional grammar and of structural and 
transformational analyses will give pupils a valid concept of how their language 
functions in conveying meaningful utterances, we still don^t know how much this 
knowledge actually helps us speak and write effectively. One modest research 
project at Coral Gables High School in Florida reports that the teachers* use of 
modern English grammar did help the experimental classes improve their ability 
to write compositions more than did the use of conventional gretmmar help pupils 
in the control classes; but the evidence is far too limited. Yet a start has been 
made in testing the relationship of a knowledge of structural grammar to ability 
to write. One major difficulty in our measuring how much students know about 
their language is that we have no tests that I know of designed to examine pupils* 
knowledge of structural aspects of the English language. 

A second controversial development, started at least as early as the 1930* s, is 
the questioning of the validity of the concept of so-called correctness of expres** 
sion* Many teachers - not all, of course - recognize the growing evidence that 
standards oi acceptable I^glish vary from class to class and from one section 
^ of the country^ to another* Research on the English actually used by educated, 
ctUtured Americans in various regions of the country has reported evidence re- 
futing much contained in widely used textbooks and items contained in tests on 
usage* We i^d tests of English usage that take into accoxmt the regional stan- 
dards of acceptance described in the linguistic atlases. 

A third development is that of finding ways of assigning English teachers a class 
and pupil load enabling them to have pupils write as much as feasible and then to 
find time to evaluate the compositions constructively^ not merely proofread them. 
Many teachers have pupils keep journals in which they write several times a week 
to supplement the writing they do for the longer compositions. As mentioned ear- 
lier, some districts are using qualified lay readers and are schediUing classes in 
patterns that allow teachers time for conferences with students. 

In addition to the problems of logistics in teaching composition, at least two others 
stand out. First, how much writing and what kinds may be beneficial? Second, 
how does a teacher establish valid criteria for judging the worth of a composition? 
You are well aware of the College Board* s experiment in using the General Com- 
position Test. You are also probably familiar with the recent research conducted 
by Paul Diederich, John French, and Sydell Carlton, all of the ETS Research Di- 
vision. They set out two years ago to •^define e^qperimentally the diverse criteria 
used by readers in deciding the merits of (college) freshman compositions. The 
bulletin, ETS DevelopmentSf for February, 1961, contains a brief report of their 
findings. The title of the piece tells us what we had suspected: Judges Disagree 
on Qualities That Characterise Good Writing* Yet teachers of English have to 
make these judgments almost every day. The problem remains. It even increas- 
-es with the introduction of lay readers into this complex, subjective process. I 
understand that the same problem of determining standards that can be somewhat 
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uniformly applied by a large number of readers or by two of them caused the demise 
of the GCT. I fear it may be doing likewise to many of us teachers of English* 

Many schools are now using the STEP tests. So far as I can tell from my work with 
the schools, teachers find these extremely useful in diagnosing and measuring stu- 
dents^ ability to discern stylistiCi rhetorical^ and grammatical merits of multiple- 
choice items. But at least two problems persist: transferring this "editorial** abil- 
ity to the composing of themes and then judging the results in a way that makes sense 
to the writer and to some other reader. 

A fourth development is the teaching of critical thinking and problem solving. Many 
olthe items in the STEP tests of writing are selected and designed to test students^ 
ability to think critically. Some teachers use the teaching of composition as a 
me atns of instructing pupils to develop the skills of critical thinking and of using the 
techniques of problem solving. When they evaluate the papers^ they look especially 
for evidence as to the kind of thinking the writer is doing. But we need tests spe- 
cifically designed to diagnose the kind of thinking students do, 

A fifth development, just now beginning, is the use of materials for programmed 
self- instruction. We need to learn what aspects of English as a cla^^sroom subject 
can effectively be reduced to programmed learning. At least one book has been 
published, English 2600 (Harcourt, Brace and Company, I960), based upon the prin- 
ciples of progran:imed learning. Several problems inhere* How can. we reduce ele- 
ments of English grammar, punctuation, capitalization, and perhag^ spelling and 
vocabulary to the detailed, itemized steps of programmed self-infitruction? How do 
we teach those who are preparing teachers and then the teachers ihemselves to de- 
velop these materials appropriately? If teachers can learn to produce these mate- 
rials tailored to their classes or can withdraw them from a bank of such items, , 
they will probably be free to devote much more time than now to important matters 
of literature and organization of compositions, style, rhetoric, diction, syntax, 
critical thinking, and problem solving* Here is an enormously important new 
field for those of us preparing teachers, for the teachers themselves, for curri c- 
ulum directors in the schools, andfor those developing tests. In the report of last 
year's Western Regional Conference are two papers presenting helpful suggestions 
on these problems: William W, TurnbulPs **Are Testing and Teaching Compatible 
and Harry F. Silberman's ^'Convergence Through Teaching Machines**. 

A fifth development is the designing of differentiated curricula for homogeneously 
grouped classes. Within these groups, teachers try to differentiate reading mate- 
rials even further. Thematic xmits facilitate this kind of planning and teaching. 
These possibili'-ies pose many problems, however. How should pupils be grouped? 
We need to have up-to-date information on the characteristics, needs, and potentials 
of pupils, English teachers particidarly need to know what values adolescents con- 
sider Important. How do we measure the relative effectiveness of homogeneous and 
heterogeneous English classes? How do we know whether we are achieving our pur- 
poses in teaching literature? How, for example, do we measure appreciation of a 
poem? How can we tell whe .her pupils are becoming lifetime readers of books? 
How do we know what effect, if any, we may be having upon pupils^ system of values? 
How do we know whether students in Advanced Placement classes do any better in 
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I college than students r -^qual ability enrolled in regular English classes throughout 

high school ?^We think chat homogeneously grouped classes have advantages, but we 
, don^t actuaily know. 

I 

Inherent in all of these developments and problems and in the related testing is the 

apparently insoluble problem of judging the competence of teachers. What is com«* 
I petent teaching of English? If it is possible to identify these competencesi can we 

then help others acquire them? How will we be able to measure, for example, the 
I effect the College Board^s Commission on English program has upon the skill of 

I teachers who study in the summer sessions? Then, how in turn do we measure the 

competence of their pupils ? 

I In summary, I wish to list the kinds of help we need in our efforts to solve some of 

the^ problems I have mentioned, I suspect that each of Hiese items has implications 
{ for people interested in constructing, administering, and analyzing tests. 

; 1. We need information from recent studies of adolescents, especially about 

the values by which they live and judge the world around them, 
i 2. We need to know what kinds of grouping of pupils will be most effective 

I for the teaching of English, 

3, We need to have better information than we now have on the relative merits 
I of the various ways of organizing and presenting the subject matter o£ Eng- 
I Hsh. 

4, V/e need to know whether flexible scheduling of classes and the use of team 
I teaching are producing the desired results. 

\ 5. We need to know whether graduates of Advanced Placement Programs ac^ 

tually do better academically in college than do students of equal ability 
I who study the regular college preparatory classes. 

I 6. We need to know something about how pupils can be helped to acquire an 

interest in and the habit of reading books. 
I 7. As pointed out in The National Interest and the Teaching of English, we 

i need to know something about **the emotional, physiological, experiential, 

and educational differences between good and poor writer s^^ inMgh school* 
I 8. We need to have the results of the experimental projects on the effective- 

' ness of the use of lay readers. 

9. We need to know how the teaching of structural linguistics to children may 
I affect the development of their command of the English language. 

10. We need to know what elements of structural linguistics may be most use- 

Iful in secondary schools and how we can teach knowledge of the English 
language that will help pupils speak and write effectively. 
11. We need to have tests of grammar that will test pupils' xmder standing of 

I the structure oi the English language and the ways in which it functions 
in conveying meaningful utterances. 
12. Colleges need English placement tests appropriate for students who have 

I been trained in modern English granmiar and not in traditional grammar. 
13. We need to know how to construct our own materials for programmed self- 
instruction and to know what aspects of English as a school subject lend 

(themselves to this kind of instruction. 
14. We need some ways of testing our impressions of students* appreciations 
of literature. ^' 
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15. We need to know how to identify and naeasure conipetences of teachers of 
English ^^d how to prepare them accordingly. 

I daresay that this catalog is already too long, so I had better st^. Some of the 
attractive qualities of English as a subject are its scope, variety, and subjectivity. 
Teachers have much room for their imagination, creativity, tastes, and prejudices. 
But, as my foregoing remarks may have indicated, this very roominess within our 
subject also invites us into treacherous wildernesses and even up onto Clouds Nine, 
Ten, Eleven. I realize that the subjectivity of our materials probably accounts for 
the lack of satisfactory measuring instruments. But I like to hope that this same 
subjectivity may sufficiently intrigue experts in testing so that they too become 
caught up in our problems and find that their only escape is through their devising 
t3sts helping us solve some of the problems I have enumerated here. 

COLONEL O'CONNOR: Thank you, Al. We now have some time for questions that 
may be addressed to Dr. Grommon and Dr. Feder. 

MRS. ESTHER G. NOLAN (Consultant in Research and Guidance, Los Angeles 
County Schools^ Los Angeles, California): I would Hke to ask Dr. Grommon if he 
knows whether the New York report stated on what basis these youngsters were two 
years below grade. Are they below grade in ability to read or in their mental ca- 
pacity in the factors of intelligence that reading requires? In the latter case it 
would be an administrative problem in modifying the curriculum and not necessa- 
rily the fault of the English teacher. 

DR. GROMMON: I fuUy support this view, of course, that the curriculum must be 
appropriate for the pupils. I thought that I had made it apparent in my other com- 
ments. Differentiating the courses and materials and expectancies, according to 
what we know about youngsters, is of great importance. I can't answer your ques- 
tion. I took this report from a recent issue of the *»New York Times'', which re- 
ported the Superintendent's speech presenting his budget to the Board of Estimates. 
This information was part of his request for more money. 

There was no information about the kind of testing given in the Nev/ York City 
Schools. Perhaps there are others here who do know. 

MRS. NOLAN: If he gets all the remedial teachers he asks for, they won't be able 
to solve all of the reading problems. 

DR. GROMMON: Well, he hasn't solved that first one, yet. 
COLONEL O'CONNOR: Another question? 

MR. ROBERT H. OYLER {Director of Guidance, Claremont Unified Schools, Clare- 
mont, California): I would like to ask Dr. Grommon if he sees any conflict between 
the current emphasis on written utterances and the current emphasis in forming 
language on oral utterances. Basically, I think I am saying: is '^speech** stiU a 
bad word with college professors? 
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DR. GROMMON: Well, maybe it appears to be eifter my previous remarks. No, I 
don'tthixik this means that the speech activities are necessarily devaluated. But 
we don't know how much -writing is necessary, and we assume from the results of 
what we now see that not e:aough writing is being done by many students. How much 
more is necessary, we donH know. I didnH go into one other aspect sufficiently. 
That is, the problem of the teachers finding enough time to grade, let's say, a set 
of papers for one-hundred-*fifty students each week. Teachers a^e driven to hxxnt^ 
ing for commas and spelling errors and turning the papers back with a grade. 

Thii» kind of reading of papers probably does not provide constructive help for stu- 
dents. But I think that as a result of the Lay**reader Program in some districts 
and the gradual reduction of load in the number of students and classes, teachers 
are finding more time to help students with their writing. Some students are writ- 
ing more. I think almost equally important is the problem of the teachers' find«- 
ing time to look at those papers hard enough to see what kinds of thought proces- 
ses are demonstrated there, and then work from this point back to help the student 
improve. But this doesn^t mean that speech is slighted, as far as I can tell. 

MRS. NOLAN: Do you see any conflict with present trends to foreign language in- 
struction as opposed to present trends in English? 

DR, GROMMON: Well, we are shifting the emphasis, as I mentioned before, to 
give enough attention to the papers, which I think were slighted. Whether this is 
going to be a conflict with foreign languages, I don't know, 

MR. SAMUEL BELLMAN (Associate Professor of English, California State Poly- 
technic College, Pomona, California): I'd like to ask Dr, Grommon this ques«- 
tion: Dr, Grommon, from the heavy emphasis that you place on structural and 
descriptive English in the new issues and problems dealing with teachers and plan- 
ners, what evidence do you have to indicate that a heavy indoctrination or a fairly 
sizable indoctrination in structural and descriptive English on the part of new teach- 
ers and professors of £^gllsh and students in high schools*«-even in college — 
will enable students to write better and to develop more effective home study read- 
programs? 

DR, GROMMON: None, yet. The only research that I have seen so far is the one 
I have seen reported in Coral Gables, and this was a very modest project. But the 
reason for this emphasis is that we know from the centuries of investigation in the 
structure of English that a good deal of what we thought of as traditional grammar 
is invalid. It is not a description of the way a Isuiguage actually functions. We are 
trying to make this rapid change in the interests of equipping the teachers with what 
we think is a valid unuer standing of the way in which our language functions and to 
convey this concept to their students. 

We hope that, through experiments and through our gaining skill in presenting struc** 
tural linguistics effectively, we will be teaching a grammar that may in time have 
some noticeable effect upon their writing, 

MR. BELLMAN: The possibility that we may go off on the wrong road — is it a mere 
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possibility? I follow the controversy and the C. C, C. Bulletin, and I wonder if per- 
haps too nrxuch investigation isn^t being placed on this. Now, I am especially inter- 
ested in the beginning of Roberts' book, Understanding Grammar, in which he states 
very candidly that he never had much success in teaching under the old grammati- 
cal dispensation. I simply wonder how much success the mosl inspired and'iinstxuo- 
tural'^ linguist will have to get students to do better work» 

DR. GROMMON: As I say, it is altogether too new. We don«t know quite how to do 
it. We are trying, but we have no evidence to answer your question. The only sup- 
port I can now give to the linguistic study of language is that it does give the student 
a valid concept of the language. Traditional grammar does not necessarily do this. 

DR. ALEX D. ALOIA (Director of Guidance Center, Loyola University, Los Angeles, 
California): I just want to make a brief statement. First of all. Dr. Feder referred 
to ^'Project Talent^» as a means of gathering data and research for our education pro- 
grams, and from the preliminary findings reported in the press it indicated that less 
than five per cent of the American youth could write a five-minute essay without mak- 
ing a mistake. The girls were better than the boys, so that ought to make the women 
feel better. 

My question is this: Since we have so much admitted difficulty in reading, English 
composition, and grammar usage, why do we persist in our schools to expose our 
children - I know why we need to - to literature, reading the *»great books*^ when 
they can^t read in many cases, and they can*t write? Is it justified? Can we de- 
fend it by having continual practice in techniques so that when they got out of high 
school they might be able to write something reasonably well? Can you respond to 
that. Dr. Grommon? 

DR. GROMMON: Well, I assvime you are not arguing that we should be content 
with illiteracy. This is my answer. 

COLONEL 0» CONNOR: We will go on. 

DR. ALOIA! May I comment on what he said? For example, I just sat down with 
a youngster the other day who is not two but five grades below his level, and he is 
now taking English literature in terms of reading, and he is also being exposed to 
a foreign language. I am assuming that in order to understand literature, we have 
to read a little bit. I agree with you, we are not content with illiteracy. But why 
do we persist in continuing to put these youngsters in *»Great Books**? Bellman 
just made a comment similar to this. Why are those who are behind put into these 
courses? Why can^t we stay with the techniques of teaching them these rules? 

DR. GROMMON: I agree. This ia the reason I emphasize the importance of 
grouping the students homogeneously. My comment is that I am not sure we know 
how to group them effectively. I have not studied Herbert Thelan^s report on his 
research on grouping* I have heard him talk about it, and I think his findings are 
going to help us a gr^at deal in learning better ways of grouping students. I favor 
homogeneous grouping. Once we can get students in more appropriate groups, we 
can then find the materials that have content of genuine interest to them and that 
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they can read with success. 

COLONEL O'CONNOR: The next speaker is the one who brought the clipping with 
him that I quoted from earlier. I might draw your attention to this. The boys say 
less emphasiSi in fact, 10 per cent of these boys say 'less emphasis should be p>ut 
on English. That is of no veilue at all. 

The next speaker is also from Coloraldo and points East, and he has been a long* 
time resident of Colorado. Professor Scherer was born in Germany and came to 
this country as quite a young man, and he received most of his education in this 
country. He attended the University of Illinois for his undergraduate work and took 
his graduate degrees, including the Ph. D. , at the University of Iowa, He was also 
a high school teacher in Iowa. He was Chairman of the Foreign Language Depart- 
ment at Stephens College in Missouri for a long while. He has been at the Univer- 
sity of Colorado since 1946 as Professor of Modern Languages. He was Director 
of the Modern Language Institute of the University of Colorado during the summers 
of *59 and '60,and is Director of the German Department at the University. 

George will speak on ^^The New Modern Foreign Language Teaching and the Need 
for Tests in All the Skills, 

THE NEW MODERN FOREIGN LANGUAGE TEACHING 
AND THE NEED FOR TESTS IN ALL THE SKILLS 

GEORGE SCHERER 

Thank you* I can't resist beginning my remarks with a story that has been going 
around in foreign language circles. It isn*t a story that has any real relevance 
herei but judging from past reports of this Conference, I have come to the conclu- 
sion that some sort of story is practically a requirement. I can only hope that the 
one I have selected is not one that you have all heard before. 

It seems that back in the isolationist twenties there was a school superintendent 
who was determined to get the foreign languages tossed out of the curriculum. To 
his surprise he found resistance among the members of his school board. So he 
invited the board to a dinner meeting at which he delivered an eloquent speech 
setting forth all of his arguments against foreign languages. He ended it up with 
the one that he felt was the real clincher; Gentlemen, he saidi ^^if English was 
good enough for Jesus Christ, it's good enough for me!** 

Since the word *'tests** appears in the title of my remarks, I must confess that I 
keep asking myself why I came here in the first place. I really know very little 
about them, so I am going to lean heavily on some criticisms I've often heard in 
the professioni and I'm thinking especially of the discussions on tests that I have 
had with Nelson Brooks of Yale. These discussions with Brooks and others have 
been frequent, for since Sputnik I have traveled a hundred thousand miles to meet- 
ings and conferences of language teachers all over the United States. Neverthe- 
less, I am still especially vulnerable when it comes to the esoteric jargon of the 
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psychometrists, probably because my colleagues don*t know this jargon either* In 
fact» I know even less about psychometry than I do about psychoceramics ** the study of 
crackpots* 

The Conference theme calls for a concentration on changes that have taken place in 
education and mer^surement since Sputnik. I take it that it is permissible to talk also 
about the changes that have not taken place but should have* Before I get into the 
changes in modern language methods, curricula and testing, since Sputnik, I feel 
strongly that a few comments must also be made about the activity in the modern 
language profession in the years between the end of World War II and Sputnik. 

In 1946 the Modern Language Association of America, through its Executive Coun- 
cil, took a hard look at reality and came up with a resolution declaring its reborn 
interest in furthering the study of languages and literatures, including English, in 
our schools and colleges. This was quite a change after the many years of almost 
exclusive concentration on the advancement of pure research* More resolutions 
followed and, in 195Z, William Riley Parker, then the Executive Secretary of the 
Modern Language Association, obtained a substantial grant from the Rockefeller 
Foundation for an investigation of the situation around us and for a bej^nning of im- 
provements. 

The Foreign Language Program of the Modern Language Association was estab- 
lished in 1946. It has since served to give the profession more and more hope. 
Through the staff in New York, aided by many able members of the profession, 
the program provided aggressive amd corrective leadership for a language teach- 
ing profession that had been seriously handicapped by many years of American 
provincialism. By 1957 the changes for the better were already quite conspicuous* 
The road to recovery had been foimd. And then Sputnik shocked the profession out 
of much of the complacency that still remained. The important result of Sputnik's 
appearance for the languages was, of course, the National Defense Education Act 
of 1958. Although some sort of a National Defense Education Act was imder dis- 
cussion even before Sputnik, there can be no doubt that this first of the man-made 
satellites assured the Act and accelerated Congressional approval. 

A number of the titles in the Act have already had a profound influence on foreign 
language teaching in the United States. Title III, with its aid to schools in the pur- 
chase of equipment, has made possible an almost frightening increase of language 
laboratories in our schools. It is something to worry about because the shiny gadg- 
ets do not always show up in places where there are teachers who know what to do 
with them. Imagine the look on the face of the grammar-translation teacher, B. A. 
Degree, 1930, 15 hours of French, whose superintendent presents her with a com- 
pletely installed language laboratory on the day after Christmas vacation! While 
laboratories, if properly used, can be of great assistance to the language teacher, 
they are certainly not indispensable. They should never be planned and purchased 
without the aid of disinterested experts, and they should never be installed without 
the assurance that so neone on the language staff knows how to coordinate class and 
laboratory work. I can see a great big boomerang in the making here. Fortunately, 
the N. D« E. A. summer and academic*year modern language institutes, plus many 
other excellent institutes and workshops, are helping our teachers to make effec- 
9^. tive use of laboratory methods. 
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Other provisions of the Act that have had their influence in the language field are the 
Title II loan program, the Title IV graduate fellowship program, and the Title VII 
program for research in the mass media of education. But the one title that has 
helped most significantly to make some of the dreams of the Modern Language Asso- 
ciation come true is Title VI, known as the Language Development Program. Its 
Chief, at first William Riley Parker, and now Kenneth Mildenberger, is the custo- 
dian of a complex, fourfold operation. There is the program for graduate fellow- 
ships in the critical languages, intended to supply the nation with a reserve of able 
linguists in everything from Chinese to Swahili. There is the complementary pro- 
gram of language and area centers, which is intended to expzmd graduate school 
opportunities for study of the critical languages and the areas where they are spoken* 

More directly related to my topic today, however, are the two other units of Title 
VI which are already revolutionizing modern language teaching in America. I'll 
speak first about the Research and Studies Unit and then about the Institute ^Unit* 

During the first two years of the Act six and one-half million dollars were obligated 
for studies, surveys, research and the development of specialized materials de- 
signed to improve modern foreign language teaching in schools and colleges. The 
i960 report of the Title VI Research and Studies Unit lists no fewer than 115 proj- 
ects then under way. In the context of my assignment here, at least three of the 
115 projects are of special fignificance: (1) The development of modern instruction- 
al materials , for grades 7 - 12, in the five most commonly taught foreign languages- 
French, German, Italian, Spemish and Russian; (2) The development of modern a- 
chievement tests, covering all of the four basic language skills in the same five 
languages; (3) The development of teacher proficiency tests, covering seven com- 
petencies, which will enable the teachers to assess themselves, in the same five 
languages. 

The instructional materials project is well under way. Under the able leadership 
of Mary Thompson a teeun of writers in five languages has already produced Level 
I of these materials - Level I being the amount of a foreign language that can be 
learned, say, in grade 9* Level I has been thoroughly field-tested, first in the 
demonstration classes of the institutes of 1959 and I960, and then during the cur- 
rent academic year in about 100 pilot schools throughout the nation. In March of 
this year Level I was turned over to a commercial publisher, Harcourt, Brace and 
World. We have the assurance of the company that the materials will be ready for 
distribution to the schools by next September. Levels II, III and IV are projected 
for the near future, so that a full four-year sequence is now assured for teachers 
who want to begin an audio-lingual program next fall. 

The new materials are called Audio-lingual Materials because the listening com- 
prehension and speaking skills only are taught at the beginning, and audio-lingualism 
remains as the fundcunental axis throughout all levels. The program is based on 
what we have learned especially from descriptive linguistics, but also from philos- 
ophy, psychology, cultural anthropology and, to be sure, from a few new discoveries 
made by the language profession itself. 

The memorization of natural dialogues is the basic principle of the new approach, 
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and ea^h dialogue is followed by drills designed to internalize the new structures of 
the language^ one at a time. The materials also enable the child to personalize the 
language used; that is, he can talk about himself. The teaching materials will have 
an entirely new format. The student manual will be of the loose-leaf notebook vari- 
ety, and the lessons will be handed out only after they have been mastered audio- 
lingually. The time-lag between such mastery and the reading of the same materials 
may vary from one week to six or more, depending upon the teacher, the situation, 
and the results of further research. 

The dialogues will be available on small discs for home study. They will also be 
aveulable on large L-P records and on tapes for the school's language laboratory, 
A series of achievement tests will be provided, and a detailed teacher's manual 
will spell out in plain English the necessary instructions for those not familiar with 
the approach. The full-blown theory behind this radical departure from convention- 
al texts is available in Nelson Brooks' recent book: Language and Language Leaxa^ 
ing, Theory and Practice, also published by Harcourt, Brace and World. 

But this pioneering venture by the U. S. Office of Education will not be the only one 
of its kind in the materials development field. Textbook publishers, knowing what 
was coming, have recently been hard at work on new language series of the type 
the profession has wanted for a long time. With tiie financial aid of the N. D. E. A. , 
the profession finally convinced the publishers that it meant business. 

A concomitant of the materials development project is the classroom progress** 
test project which is in the hands of the best foreign language testing expert in our 
profession. Nelson Brooks. Under his direction, sixty people have been working 
on item-writing, with the assistance of many technical experts from ETS and par- 
ticularly from its Cooperative Test Division. The tests will probably be known as 
the MLA Cooperative Progress Tests. 

Tests are somewhat like Easter hats--they range from captivating to exasperating. 
This is a slight revision of a turn of phrase coined by Nelson Brooks and used in 
another connection. He said, captivating to nauseating**, but 1 want to go home in 
a jet, sitting up, smd not in a coffin, lying down. The coming Brooks tests are 
among the most captivating tests I know. They are already being pre-pre-tested 
in a few schools. They will be pre-tested on a grand scale next year, and they will 
be ready by 1963. They will be made for two levels: level one and a half and level 
three and a half, and in two forms. They will cover all of the language skills-- 
listening, speaking, reading and writing--in five different languages. The norms 
will be set up on the basis of the types of schools in which the tests are pre-tested. 
It is hoped that this series of tests will take the place of the old Cooperative Tests, 
which are now severely outdated* 

There is another series of captivating tests known as the Modern Lfanguage Asso** 
ciation's Foreign Language Proficiency Tests for Teachers and Advanced Students, 
which are being developed with the active collaboration of ETS, and \mder the bril- 
liant leadership of Wilmarth Starr of New York University. The tests cover the 
seven competencies that the profession has identified as being essential to good 
^ language teaching: listening comprehension, speaking, reading, writing, language 
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analysis, culture- civilization, and professional preparation. The tests are going 
to be available in five language fields and in two forms. They were thoroughly pre- 
tested both at the beginning and the end of the I960 N. D. E. A. sununer institutes, 
and in a few additional control programs. In a March, 1961 newsletter the direc- 
tor, Wilmarth Starr, said the following: ''Among the uses contemplated are place- 
ment, diagnostic and proficiency measurement in graduate programs and very prob- 
ably in imdergraduate major programs at advanced levels. Several institutions 
have indicated an interest in certain of the tests in connection witix graduate Ian* 
guage requirements. A major use is anticipated as a result of expressed interest 
on the part of State Certifying Agencies in cozmection with or as a supplemeni; to 
State Certification Requirements. Revised forms of these tests will be adininis- 
tered in the 1961 summer institutes, and at the end of the summer standards and 
norms will be set up so that the transition to general use can be made. We are con- 
fident that these tests will work well for some time to come. 

This brings me to the language institutes of the Language Development Program, 
the fourth of the programs of which Kenneth Mildenberger is the chief custodiax^. 
In 1959 almost 1, 000 modem language teachers from our elementary and secon- 
dary schools were given the opportxmity to upgrade themselves in the seven teacher 
competencies I mentioned before. In I960 some 2$ 000 additional teachers had the 
same opportunity. In 1961, about 4, ODD more of our language teachers will attend 
the institutes. The implications for change in the classroom are, of course, ob- 
vious. Most of the 7, 000 gradixates of institutes who will be teaching next fall will 
be using materials of the type I described before. They will be using the new ap- 
proach because they will have been trained for it, through demonstration classes, 
by expert high school and grade school teachers of languages. From the favorable 
feedback that is coming to the directors of the institutes, we know that most insti- 
tute graduates actually do adopt the new approaches. Heretofore, they have been 
somewhat frustrated, however, because new materials beyond the first few units 
were not available to them. In desperation many have produced their own. The 
7, 000 re-trained teachers that we will h^ve by September r^resent a substantial 
segment of the profession. We know, fo^ example, from one of the surveys made 
by the Modern Language Association's Research Center, that there are about 
26, 000 secondary school teachers of modern foreign languages in the nation. We 
also know tixat many of the institute graduates are in influential positions as de- 
partment heads, system and county supervisors, and consultants in the state de- 
partments of education. They are energetically at work re-traixiing their many 
colleagues in the newer techniques. Add to all of this the fact that we have always 
had some teachers who never gave up the direct method in spite of the difficulties 
of accomplishing much with it in what was, until recently, a standard two-year 
terminal sequence. And, of course, some of our colleges are finally waking up, 
so that many new teachers are now getting the kind of training the institutes are 
providing for those in service. At the University of Colorado, for instance, al- 
1 most all beginning language courses in ten different l.^^guages will be taught audio- 

lingually by next fall. 

I In the professional revolution that I have tried to describe there is a clear and vi- 

brant cry for new tests that are compatible with the newer teaching approaches. 
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These tests must cover all the skills, for we have learned that none can be tested 
indirectly. If the national tests of the various prestige agencies do not become rel- 
evant to the newer teaching, great damage will be done. In fact, much damage has 
already been done because many of the re-trained teachers are forced— or think 
they are forced— to pollute their audio-lingual approach with drill in grammax-trans- 
lation. The tests, in other words, are seriously blocking progress. 

Let us look at some of these instruments that now determine the fate of thousands of 
students each year who are seeking free trips to Europe, entrance to college, ad- 
vanced placement, or acceptance in graduate schools. I mention the free trips to 
Europe first because I have an unofficial confession to make for the profession. We 
have a series of national organizations known as the AAT's, that is, the American 

Association of Teachers of , and you can £IU in almost any common language. 

Some of these AAT's are now sponsoring national contests for high school students, 
and the winners get attractive prizes, such as jet flights to Europe. But you should 
see the tests ! They are excellent examples of the exasperating variety. To begin 
with, they are very deceptive in that they look as though they were standardized, but 
in actuality they are pre-tested only to a very Umited degree, they are not subjected 
to an item analysis, and they are not normed. There is neither any uniform control 
over the conditions under which they are given nor over the manner of scoring. What 
is even worse, they foster everything except audio-linguaUsm, the most important 
training for a student who is going to profit from travel abroad. In preparing stu- 
dents for these examinations teachers must concentrate on grammar and translation. 
As one student said, "I had two years of Basic French and when I got to Paris I 
couldn't find anyone who spoke Basic French. " It has been wisely suggested that 
the AAT's now sponsoring these contests, and those contemplating similar ones, call 
a moratorium on the written tests and substitute a variety of audio-lingual situations. 
These might well be interviews, debates, skits or declamations, all in the foreign 
language. 

And what about the College Boards? They are even more exasperating because they 
have so much prestige. No one can really force a student to participate in a con- 
test, but if he wants to go to a good college he may have to submit to the College 
Boards, and on the teacher rests the responsibility for his success. So here we 
go again. Time must be stolen from audio-lingualism to cram for what is in the 
College Boards, or what the teacher thinks is in the College Boards, for the teach- 
er has only a vague notion as to what they are like. The last complete test was 
distributed in 1941. So the teacher now teaches toward the small sample that is 
provided and toward his own assumptions as to what the total test must be like. Al- 
though our spies tell us that the student is no longer asked to translate, the teacher 
is not sure of this. Knowing the tradition of the College Boards, the teacher takes 
no chances and drills his students in grammar-translation, that is, translation 
from English to the foreign language. The College Boards, of course, still weight 
the score heavily in favor of the literary language. While the profession is by no 
means forgetting about the visual and the graphic aspects of language, it feels that 
there must be an adequate representation of the spoken style in any such test. 

The recent addition of a listening comprehension test to the College Boards was 
most welcome. But it was made optional. To complicate the matter it was 
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administered in the schools under test conditions that varied from very good to very- 
bad. This, of course, generated much justified criticism. If it is impossible to 
give the entire battery in controlled test centers, then it would seem more judicious 
to give the written tests in the schools and the listening comprehension tests, as 
well as the hoped-for speaking tests, in the testing center St Adequate control over 
the written examinations is possible in almost any school, but for the audio-lingual 
tests this is at present out of the question. 

Let us take a quick look at the Advanced Placement Examinations of the College 
Board, which now include a listening comprehension section. On the whole, this 
section is quite commendable. However, in some of the examinations which I have 
seen there is exhibited a disturbing view of what listening comprehension really is. 
This is a skill which can very easily be tested purely. By this I mecUi that in order 
to test for listening ability one does not have to include reading. Whenever the 
stimulus is given on tape and the multiple choice answers are before the student to 
be read we do not have a pure listening test. The student who canH read will fail 
the test, even if he understands the spoken language perfectly. We do not have to 
give up the machine- scor able answer sheets in order to make the test pure. There 
is the simple expedient of taping the possible responses as well as the stimuli. If 
someone should say that this might then become a test of memorization rather than 
of French, I would suggest that the stimuli £tnd the possible responses b'^ played 
twice. But in order to increase the difficulty of the test, the items might well be 
played only once in the latter part of the test. The criticism that testing condi- 
tions vary exceedingly from school to school can be taken care of by the use of the 
testing centers. 

Among the more exasperating tests are the language tests in the Teacher Educa- 
tion Examination Program. They have no listening or speaking sections, so that 
they do not tell us very much about the candidate's potential as a teacher of lan- 
guage as communication. These tests were normed chiefly in teachers' colleges, 
and this fact probably explains why they are not very discriminating. Good can- 
didates for graauate work find themselves hitting the ceiling of these tests with 
their heads. Probably the main reason that good students keep wanting to take 
the tests is that tney offer a readily obtainable high score for the record. Unless 
these tests are to be replaced by the Modern Language Association Foreign Lan- 
guage Proficiency Tests for Teachers and Advanced Students, the language tests 
in the Teacher Education Examination Program will have to be completely over- 
hauled and expanded to cover adl the skills. The new versions should also be de- 
vised so as to provide for part scores for diagnostic purposes. 

The Graduate Record Examinations are not very enlightened either. The main 
trouble with these examinations in the language field is that they put too much 
emphasis on literature as opposed to language skills. It is a known fact that The 
Graduate Record Examinations do not show whether or not a student is prepared 
for graduate work. I should like to give two reasons for this assertion. First of 
all, in many graduate schools the literature courses are conducted in the fo?*eign 
language. While it is important to know how good a background in literaturi: a stu- 
dent has and whether or not he has ability in literary criticism, we must alr.o 
know whether he controls the spoken language well enough to follow the lectures 
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and participate in the discussions. Furthermore, we subsidize many of our grad- 
uate students by giving them beginning language courses to teach, so it is doubly 
important that they be well versed in ^ of the language skills. 

The second reason for the criticism of this test is that foreigners taking it have a 
great deal of difficulty with fixed- response tests simply because they are not ac- 
customed to them. This is such a very important matter in the foreign language 
field because we have many non- Americans in it. In addition, the non- Americans 
working in the field of a language and literature which is native to them have no op- 
portunity to show their superior audio-lingual ability. 

In view of the failure of The Graduate Record Examinations to provide the informa- 
tion we need, we are left to our own devices for determining the ability of students 
to participate in graduate course work given in the foreign language and also their 
ability to teach beginners with an audio-lingual approach. We have the choice of 
interviewing them or of requiring them to submit, along with the rest of their cre- 
dentials, a tape recoiding of some material in the foreign language in which they 
wish to work. The interview is, of course, the better technique, but it is usually 
not very practical. The tape recording is not a guarantee that the student has suf- 
ficient fluency in the language, but it does help in determining voice quality and the 
degree of accuracy in phonology. There may be other ways by which one could 
solve this problem but the simple way would be to include listening and speaking 
sections in the Graduate Record Examinations. 

The construction of relevant tests that cover the target discipline fairly and com- 
pletely depends in large measure on the appointed committees and the inner com- 
patibility between the technical experts and the subject-matter people. There are 
many bright people at the test production centers. There are also quite a few in 
the language and literature profession. The two groups need each other badly. 
Some of the bright professors can write good test items and some simply can't. 
Some of the testing experts have more talent for working with one subject-matter 
group than with another. It has been suggested that the professors being consid- 
ered for a committee assignment might be asked to submit a few test items be- 
fore a commitment is made. This sounds like a workable idea. By the same to- 
ken, I would suggest that the psychometrists being considered for the same assign- 
ment be checked somehow for their degree of ability to see the problems of the 
profession. 

It is worth noting that professors with national reputations for their teaching and 
research are not necessarily the best test-itejn writers. This business i-equires 
a special talent. Sometimes the relatively uj^nown assistant professors have this 
talent. We ought to attempt to identify more of these people in the future. 

In addition, the strenuous effort usually mad^' to get geographic representation on 
every test committee is really unwarranted. ) It isn't very wise to sacrifice a good 
item writer because he lives and works in th6 same region as another. For all I 
care, if the best half-dozen people for a certain committee all happen to live in ' 
Rhode Island, let's sign them up. Think of how much cheaper it would be to get 
them all together ! 
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I have heard it suggested that we need a high-level commission in each of the sub- 
ject-matter fieldsi a commission that could advise ETS and all of the other organ- 
izations that publish and distribute tests concerning the needs in new tests, the 
disposition of old tests, etc. This, it seems to me, is an admirable suggestion* 

Those who only a few years ago were convinced that the teaching of modern for- 
eign languages was headed for the graveyard are now aware that this was self- 
deception. The jets have annihilated distance. We are involved in affairs all a- 
round the globe. International TV will soon be a reality, and so will the Peace 
Corps. Language for communication is indispensable in our time* This is the 
one chief reason for audio- lingual approaches. 

No one among the audio-lingualists whom I know wants to neglect literature. On 
the contrary, they know that an audio-lingual ability in the language will msike the 
subsequent visual-graphic study of literature far more meaningful than ever be- 
fore, and this is the other chief reason for audio-lingual approaches. 

Language in the air is now just as importzuit as language on papor and that is why 
I have talked so much about the missing audio-lingual tests. V/e now know how to 
make them so there^s no longer any real reason for omitting them. Unfortunately, 
the speaking tests cannot follow the happy pattern of pencil marks on a sheet of pa- 
per. But they ar^ badly needed everywhere, ip education, in government, in in- 
dustry and in research. And if it should be suggested that the teachers will teach 
toward them, so much the better. 

COLONEL O'CONNOR: Very fine, George. I enjoyed it, and the audience will, I 
am sure, appreciate having a written copy of your remarks. I would venture a 
guess that the teaching of languages has improved more remarkably since World 
War II than teaching in any other subject matter area. 

I watched some measurement going on at my own institution last week* . He end-of- 
the-year exctmination of all Cadets was taking place in each foreign language, and 
it was entirely verbal* Every Cadet came before four instructors seated around a 
table and talked with them in the language he had been studying* No English was 
used whatsoever. 

Now niay we have some questions for George Scherer? 

MR. T. LESLIE MACMITCHELL (Director, College Entrance Examination Board, 
Western Regional Office, Palo Alto, California): Dr. Scherer, perhaps the first 
comment I ought to make is a humorous one, "It is nice to be among friends, even 
if they are not your own* " Dr. Feder, in his paper, made reference to prominently 
known people, specifically Dr. Holt and Dr. Tyler who took some statistics on test- 
ing and made two entirely different deductions from them. Dr. Holt felt that schools 
are very much concerned about testing; Dr* Tyler felt there is no serious problem* 
Dr. Holt, as Dr« Scherer suggests, felt that teachers are teaching to tests. I sug- 
gest that this is a criticism of teachers, rather than of tests* 

Any teacher who takes the trouble to read the descriptions of the language tests in 
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the descriptive booklet will have a very clear notion of what the tests are like* It is 
difficult to see how letting a teacher see a single complete test would help her appre- 
ciably in better preparing her students for another form of the test, and the educational 
soundness of teaching specifically for a test would be questioned by most teachers. 

In connection with College Board Achievement Tests, which include foreign language 
testing of 350 College Board member colleges, about a third of them require one or 
more Achievement Tests* The majority of these colleges require them for place- 
ment purposes, not admission* 

The University System of Oregon, for example, has just moved to a requirement of 
Achievement Tests for all entering students for placement purposes. At the Univer- 
sity of Oregon, the test in foreign leuiguage is specifically required. 

Several otuer comments are in order* First, the purpose of the Achievement Tests 
is to test what is b,eing taught in the secondary schools* If the secondary schools are 
not teaching by the audio-lingual method, it will do us no good to test this particular 
skill* As soon as the fine work being done in this area by the Modern Language As* 
sociation is accepted and used in the schools» the College Board Examiners will re- 
spond. We don't want ceachers to teach to the tests, we wetnt to test what is being 
taught* 

Second, listening comprehension work and listening comprehension tests are gain- 
ing in popularity* Rarely do I visit a college or school that does not have a language 
laboratory, of which it is very proud. However, I know of few colleges that are re- 
quiring listening comprehension tests of entering students* I do know of many who 
would like to and, hence, I believe we will see a considerable growth in listening 
comprehension test requirements in the next several years* 

Third, it was suggested that these tests should be administered at national centers, 
instead of in the schools. They are not offered at national centers because there is 
not sufficient demand for such testing, and the equipment needed in testing is ex- 
tremely expensive. For the past four years a committee has been working on this 
problem, that is, to obtain audio equipment which would do the job effectively and 
yet at relatively low cost. 

When it comes to providing tests of listening comprehension, the Board and ETS 
have struggled hard, against formidable practical difficulties, to come up with 
workable solutions, and no one knows this better than Nelson Brooks, The Board 
and ETS have been active from the start in the development of listening compre- 
hension tests. Nelson Brooks was a member of an ad hoc committee appointed by 
the Board to explore ways of administering these tests at centers. After seeking 
out and reviewing the best advice available on all possible arrangements, the com- 
mittee agreed there was no satisfactory solution that would cost less than half a 
million dollars. Listening comprehension tests were developed, however, and 
were introduced in the regular Placement Program, and in the Advanced Place- 
ment Program, Their introduction in the Supplementary Program last year rep- 
resented a resourceful solution of the problem of somehow fitting them into the 
Board^s admissions testing program. The fact that they are administered only at 
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schools which choose to administer them, and under varying conditions makes it 
imperative from the Board's point of view that they be regarded as optional. 

Fourth, our Achievement Tests in foreign languages are made up by school and col- 
lege teachers - made up to reflect what is being taught in the schools. It seems to 
me, therefore, that if teachers are exposed to the new theories of the Modern Lan- 
guage Association and accept and use them, it will not be long before our commit- 
tees will begin to introduce these new techniques into the tests. 

In summary, there are several difficulties here for the Board. It is the function 
of each subject-matter examining committee to prepare a test that will be as fair 
as possible to all csmdidates, coming from different schools and taught by teachers 
with somewhat varying objectives, using a variety of methods and materials. Many 
teachers are now using the audio-lingual approach, as Dr. Scherer indicates, but 
many are not. In fact, some are opposed to the approach. The community of school 
and college language teachers is by no means united on the question of the role lis- 
tening and speaking skills should play in the teaching of language. The College Board 
examiners have prudently taken a neutral position. 

COLONEL O* CONNOR: That question was well put. We expected that from Les 
MacMitchell, and I am sure everyone has appreciated it, and I am sure Georgewas 
angling for it, too. Any other responses here? 

DR. WILLIAM M. SLAI CHERT (Associate Professor of Education, Denver Univer- 
sity, Denver, Colorado): Would it be possible or desirable for the English teachers 
and the foreign language teachers to get together in the teaching of language? We 
will ask that of Dr. Scherer, the fellow from up the Turnpike way. 

DR. SCHERER: They stlready have. In March we had a meeting at Indiana Univer- 
sity . There were present eleven well-known Chairmen of English Departments 
and eleven Directors of Foreign Language Institutes, and this was the whole idea 
of the three-day meeting. Among those present were the men I spoke of - Nelson 
Brooks, William Riley Parker, Kenneth Mildenberger, men from the Institute Unit 
in Washington, and many other resource people; and we discussed for three days 
the possibility of organizing institutes for teachers of English, sponsored under 
the N. D. E. A. 

After the first day - maybe I should say after the first evening over the bourbon - 
these English boys got pretty well warmed up to the whole situation. At the begin- 
ning I think they felt that the people in Modern Languages really couldnH be very 
helpful, but they listened attentively, asked many searching questions, and I think 
they went home with a great many ideas about organizing institutes for teachers 
of English. I hope that out of all of it will come, first, some money from Congress 
for N. D* E» A. English Language Institutes and, secondly, a good program. 

DR. GROMMON: May I comment briefly on this question? Certainly the National 
Council and the other groups of English teachers are welcoming this kind of co- 
operation. In connection with an earlier question about language versus structur- 
al linguistics, I should report that until recently one of the strongest forces 
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requiring English teachers to teach traditional grammar was the teachers of foreign 
languages. They insisted on traditional English grammar's being taught so they 
wouldn't have to teach so much of it. Now the reverse is true. Many teachers of 
foreign language are teaching linguistic concepts of language. Whereas before they 
were forcing us to teach traditional grammar, now they are leading us in teaching 
structural linguistics. 

COLONEL O'CONNOR: We have not heard from the science people yet, and we 
have a lot to learn. We will adjourn now until 1:45 p. m. 

(The morning session was adjourned at 12:00 noon. ) 
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The Tenth Annual Meeting of the Western Regional Conference on Testing Problems 
on "Changes in Education and Measurement Since Sputnik, " sponsored by the Los 
Angeles office of the Educational Testing" Service, convened at 2:05 p. m. in the 
Blossom Room of the Hollywood Roosevelt Hotel, Los Angeles, California, Gener- 
al Chairman Colonel Virgil J. O'Connor presiding. 

COLONEL O'CONNOR: We have two speakers this afternoon, and we will start at 
the appointed time. At 3:30 you can leave. We don't know .whether the speaker 
will stop or not, but you can leave. 

The Xirst speaker this afternoon is also a neighbor of yours from the North. 
Dr. John L. Kelley did his andergraduate work at UCLA. He was born in Kansas 
and received his Ph. D. at the University of Virginia. He served time at the Insti- 
tute for Advanced Study at Princeton and was a Fulbright Professor at Cambridge. 
In addition, he for a time graced the math departments at both the University of 
Chicago and Notre Dame. He is now a Professo of Mathematics at the Univer- 
sity of California at Berkeley. He is a member of the Committee on the Under- 
graduate Program of the Mathematical Association of America. Among the books 
to his credit is Introduction to Modern Algebra, which served as the text for the 
widely viewed Continental Classroom television series on modern algebra. Dr. 
Kelley' s charm and gifted presentation of this series have made him the most wide- 
ly known mathematics prof^-asor in the United States. 

If I tell you the story on hitn, you will know it is not true, but it is about his way 
with students. This young fellow, like Dr. Kelley l:ere, wanted everything to be 
proved, and he was always interrupting: "Well, can you prove it?" One day he 
wanted to see the document that Pmfessor Kelley had referred to and Professor 
Kelley said, "It is in my files somewhere. I am quoting from it. I will show it to 
you sometime if you want to see it. " The kid said, "In the meantime, until I see 
it, is it all right if I call you a liar?" The Professor didn't stop him. He said, 
"Young fellow, I assume your parents are married. " The poor kid said, "Why, 
of course, sir. " Oh, I can see you are with me this alternoon - I guess you can 
fi^u/fc 1 , u the rest of the story. 

I will let Professor John Kelley, f-acher of modern mathematics, proceed. 

THE TEACHING OF MODERN MATHEMATICS 
JOHN L. KELLEY 

We were just comparing notes at lunch. My great grandfather came from Ireland 
during the potato famine, and we don't think that our great grandfathers were on the 
same .ihip. They missed by a couple of years. 

I am supposed to talk on "Changes of Education and Measurement Since Sputnik, " and 
in particular on the "Teaching of Modern Mathematics. " Perhaps I am here under 



38 



false pretenses, oecause I will have very little to say about changes since Sputnik, 
although sorne of the things I will be talking about are changes that have occurred 
since World V/ar 11. I will have very little tc • ay about testing, but the things I 
want to talk about have something to do with testing. I think, instead, what I will 
try to do is say something about modern mathematics. In a highly organized soci- 
ety, we have such specialized jobs that it is frequently difficult to explain to some- 
one else what you do. If you say you are an economist, a chap may ask you to help 
him with his income tax; if you say you are a mathematician, you are likely to be 
asked to add the bridge score. So, I want to talk about my job, which is mathe- 
matics, and about the change in mathematics, which has been called lately by Mar- 
shall Stone, the ''Revolution in Matheniaucs. " I am a little bit afraid of the term 
revoluti oil » since the Cuban episode, but I guess I can talk about it. 

One of the things that many of our students do not realize is that mathematics is a 
changing, growing discipline. New mathematics is being invented every day and 
at a rate which is downright appalling. I will try to give you some figures to ex- 
plain what I mean by this: There is e review magazine called Mathematical Re- 
views, which gives a short summary of new mathematics as it is printed. Nor- 
mally, an article - a research article « which is perhaps ten or thirty pages will 
be reviewed in something between two and five inches of type. In 1940, the num- 
ber of pages in a yearns issue of Mathematical Review was four hundred. That 
isn^t the number of pages of new research; that^s the number of pages of reviews 
of new research. Many people will find this shocking. 

In I960 the same journal, by the time they published the last volume - and they 
are still missing one issue of I960 - will have eighteen hundred pages devoted to 
review of new mathematical research. What is even more shocking, our best 
estimates are that in 1962 there will be four thousand pages in Mathematical Re - 
views. There has been a tremendous explosion of mathematical research and in- 
vention, which is one of the things I wanted to tell you about. 

There are many new mathematical subjects and new applications. By request, I 
will tell you a story concerning a new subject, a new theorem. Many people feel 
that mathematics ended, at the latest, with Newton, and I have often been asked 
to give an example of a new mathematical theorem which has been proved in this 
century. So I will give you such an example. 

Most mathematical theorems require quite a bit of mathematics to understand 
them. It is like listening to someone repairing a radio - '^circuits, »»tubes, 
•'resistors'* - you can't make it out. So I will tell you in very informal terms a 
new theorem. Consider the following situation: You have a cue ball and, by some 
great good fortune, you were able to grow hair on this billiard ball. You are now 
in possession of a hairy billiard ball. For some reason you are^eeling tidy, and 
you want to comb the hair on this billiard ball, but that isn't all - you want to comb 
it straight, with no part, no cowlick, or no swirl. This, then, is the problem 
which you have posed for yourself. You have the furry, hairy billiard ball, and 
you want to comb it tidily, wita no part or no cowlick. Now, it is a theorem of 
this century that you can't. This is a rough statement of that theorem. If I stated 
it mathematically, I would say there is no non- singular vector field on a sphere, 



and you wouldn't like this. I believe, if you think about it a few minutes, you will see 
that the theorem is probably within reason. If you try to keep combing the hair all 
the way around the cue ball, you are going to have trouble. If you grow hair on a 
doughnut, the problem is much simpler, because there is not one, but several, ways 
of combing this. This is a very informal statement, but this is one of the new the- 
orems which is fairly easy to describe in nontechnical terms; and there is some con- 
tent to the theorem, even though I have described it in this facetious fashion. This 
is one bit of new mathematics. 

I said, and I want to go back to this, that there are both n**w subjects and new appli- 
cations. Let me talk a moment or two about some subjects which are quite new. 
Let us suppose now that we are thinking from the point of view of applications. Sup- 
pose that you are a manufacturer, and you manufacture fuses to put in the circuits 
of your house. You don't want to manufacture faulty fuses, because the house may 
burn down; so you want to test the fuses* Now, there is a very simple way to test 
a 15 amp fuse. You put it in an overloaded circuit, and if it burns out, it was a 
good fuse. You see the point is this: if you test the thing you can't sell it. There 
are many, many things which are of this character, where a test is self-destruc- 
tive. Almost all new things are of this character, many things which are less pro- 
saic than a fuse out of a fuse box. How do you want to test them? Obviously, you 
want to sample, don't you? You test a few samples, but there are some fairly 
serious problems. Let's say that you are making a large number of these* If you 
do not sample enough to be sure that nothing has gone wrong with the manufactur- 
ing process, you run the risk of turning out faulty and perhaps dangerous mer- 
chandise. On the other hand, if you sample too much, you run up the price un- 
necessarily, and you may r\m yourself out of business. 

The problem then: What sort of sampling procedure are you going to need if you 
want to be 99 per cent sure the items you let through are all right? This is the 
problem of quality control, and there are ^several thousand people working on it in 
this country. It has been developed highly during and since the war. Now, the 
mathematical background for quality control is statistics and probability theory, 
and this is one of the bits of mathematics which is gradually filtering down from 
the university curriculum into the high school curriculum. One of the recom- 
mendations of the Commission on Mathematics of the College Entrance Examin- 
ation Board was that probability and statistics should be offered in certain programs 
in the senior year of high school. This is new mathematics- -new mathematics in- 
vented to meet a new situation. 

I might give another exeimple also having to do with statistics: There are three 
telephone companies in Ohio, and on a long distance csM it is likely that you will 
use the lines of two or even all three of the companies. Now, it is perfectly clear 
how the revenue ought to be divided. You see where the call originated, and you 
give each company a proportion of the charge, depending on the proportion of its 
line that was used. There is only one trouble: making l^is computation is extreme- 
ly tedious and it costs money. Last fall, before a Public Utilities Board, these 
companies asked to be permitted to divide the revenue on the following basis: There 
are an enormous number of calls. They proposed to choose a sample of these calls 
and split the revenue precisely on the calls taken in this sample, and then simply 
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divide the total revenue in the proportion that the sample indicated. It saves money; 
it saves money for the company and for the people who use the telephone. This is 
large-scale use of computing. 

I might give another example of the sort of thing that causes increased use of math- 
ematics, and there has been increased usje. I remember a friend of mine once ex- 
plained to me that the first World War was a chemist* s war and if you think of the 
developments in high explosives and gas and such, perhaps there is something to be 
said for this. For evident reasons, the second World War could be called a phys- 
icist's war. There was more highly complicated equipment used, as many of the 
new advances were more physical in character. It is predicted that the third and 
last World War will be a mathematician's war, which is a gruesome thought. 

I want to speak briefly about autoniation. There is nothing very complicated about 
automation. Fimdamentally, if one machine controls a second machine, this is 
automation. You all know about automation. You have it in your kitchen; you do 
not turn the refrigerator motor on and off every time you want to use the refriger- 
ator* This is done by another machine called a thermostat. The thing which is 
different is the degree of automation - and let me give an example. The calcula- 
tions that are required to fire a ballistic missile are complex, but certainly not 
beyond the ability of people to do it. They are the kind of calculations that com- 
puters can do; and one might say, **Why worry about automation in this respect?" 
You know the old story about automation eind engineering in China: A Chinese 
engineer was explaining to an American engineer that - well, Americans took 
things much too seriously. »» We want to build a tunnel through a mountain. It is 
very simple. We put 5, 000 men on one side of the mountain and 5, 000 men on the 
other side, and we set them to work. " "But you worry about whether they meet or 
not, don't you?" "If they meet we have a tunnel; if they don't meet, we have two 
tunnels." Now, with intercontinental bedlistic missiles, you have computations of 
the sort that can be done by people without machines. You might say, "Why not 
get 9» 000 people to do them instead of building these fantastic machines?" and it 
is very easy to explain why not. You see, when the missile is fired, the primary 
control is while the motor is burning, and that's only a few minutes, not very long. 
During that time one has to have information on what the missile is doing. You 
must have an estimate of what is going wrong and how to correct it. This infor- 
mation must be received, and the computations must be made of the necessary 
correction; and feedback must be sent to the missile in the initial few minutes of 
the firing. This is the sort of calculation, then, that has to be done in such an 
incredible hurry that, even though it is well within the possibilities of a gang of 
computers, it can't be done, because of time, by human computers. This is a 
rather advanced form of automation. You have to have the machine to tell the bal- 
listic missile what to do. 

The ability of these computing machines is rather fantastic, and I am sure you have 
heard stories; but let me tell you one or two more: During the nineteenth century 
an English mathematician - I think it was Shanks - decided he would compute pi, 
the ratio of the circumference of a circle to the diameter. He spent about twenty 
years at it, and he computed, I believe, about seven hundred and some decimal 
places. It was his way of amusing himself. Now, shortly after the war, on the 



first of the high speed electronic computing tnachinesi a tnathematicisin from the 
University of California was able to get a weekend of machine time - about seventy 
hours- and among other things, he computed pi to Z, 000 decimal places* Inciden- 
tally, the chap Shanks was wrong from about the 500th place on. That was right 
after the war, and the difference between twenty years and a weekend is, roughly, 
the difference made by the equipment. Since that time much better computing 
machines have been made and the first 2, 000 places of several constants, pi in 
particular, have been computed in about fourteen minutes. Probably the record is 
below that, but people have lost interest in this kind of thing. 

Machines that run machines have had a tremendous stimulus mathematically, and 
I will say in a minute what this has to do with teaching and testing. We really are 
in a sort of push-button age, and I think many of you may have seen a little gadget 
on the market now. It is a little black bo:c v;ith the switch on the side of it. You 
push the switch down and the motor goes on, and the lid opens, and a hand comes 
out and turns the switch off, goes back in and the lid closes. This is a very nice 
example of the push-button age; but I heard a better one the other day: This is a 
push-button which is really needed* This push-button has no light that flashes, no 
motor that runs, no noise. In fact, nothing happens, except this: You push on the 
button and the button pushes back and makes you feel wanted* 

There are other excunples of new problems which make new mathematics neces- 
sary; but I think I will leave these and say what this has to do with the teaching of 
mathematics and the testing of mathematics. The mathematical curriculum has 
changed drastically* As was mentioned earlier, I belong to the Committee on the 
Undergraduate Program of the Mathematical Association of America, and this Com- 
mittee has worked for rather more than ten years trying to improve the mathemat- 
ical curriculum in the colleges. The difference that Sputzuk has made in these en- 
deavors cjQd these developments is not in their character. The changes in curric- 
ulum and the invention of modern mathematics went on before Sputnik; the Commit- 
tee on the Undergraduate Program was in existence and working several years 
before Sputnik. But now, having said this, what Sputnik has done is to make pos- 
sible large- scetle experimentation and curriculum study. The School Mathematics 
Study Group is one of the examples. There are also groups at Illinois, Chicago, 
and I don't know where else. These people are concerned with experimenting and 
improving the curriculum, and Sputnik has made it so you can get money to sup- 
port these activities. I mean you can't write a new textbook or make a new pro- 
gram without financing it, and this is the thing which has changed since Sputnik in 
the teaching of mathematics. 

I might digress to say - because I was asked already today - how good the Russian 
mathematicians are. I must say many people have asked me this, and the answer 
is: the Russian mathematicians are first-rate. Their mathematical work is pos- 
sibly ahead of their work in physics and other directions. There is nothing to be 
gained by presuming that we are ahead of them mathematically. We are not. They 
are just as good as we are. We get no comfort from thinking of the international 
situation and our mathematical resources. Well, let's return to the chsuiges in 
mathematics. 
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There have been remarkable changes in the teaching of mathematics in the past 
years, and I think I may illustrate these by relating some of my own experience, 
I recently wrote a book on modern algebra, which was intended for college fresh- 
men or talented high school seniors. Perhaps this was effrontery on my part, 
because I never took such a course or, in fact, any course in modern ctlgebra. It 
is remarkable that some of the material which now occurs at this level - college 
freshmen and accelerated classes in high school - is material which was taught in 
the junior year of college sixteen years ago, and in the graduate years of college 
twenty-five years ago. It is new mathematics for the level at which we are now 
teaching it* I might mention another example. I never took a course in probabil- 
ity and statistics. Closely connected with probability is measure theory. This 
was a graduate subject when I was a graduate student, and, as a matter of fact, it 
was not universally required in the graduate school. Now, at the University of Cal- 
ifornia and many other schools, a course in measure theory is a standard junior 
course in mathematics - a third-year course. The situation is this: A mathemat- 
ical topic seems to work itself to a lower and lower grade level. Things which 
were taught in the graduate school are taught in the college years. Things which 
were taught in the college yeSts are now being taught in the high school years. 

It may well be asked how we manage to add these subjects to the curriculum when 
no more time is added to mathematical instruction. It is well loxown that you can 
stuff only so much paper in one wastebasket, and it seems reasonable to expect 
that you can stuff only so much information into the students in a fixed period. 
This brings us to the question of the changes made in mathematical inatruction and, 
in particular, to the changes made in teaching more or less classical mathematics. 
I want to talk about these in some detail but, before doing so, I want eo mention one 
important reason that we in the universities are able to teach more mathematics to 
our students than previously. University students are better prepared than they 
were. I know it is popular to complain about the quality of entering freshmen. It 
is one of the favorite collegiate pastimes of professors. But I am positive - and I 
believe this is born out by tests - that entering college students are better prepared 
in mathematics tham they were ten years ago or fifteen years ago. They are better 
prepared. 

Let us return to the question of how the mathematics curriculum improves. One 
way it improves is that we find better methods to do things. Those of you who have 
studied calculus may have tried to look at Newton* s Principi a. If you did, I hope 
you were not as confused as I when I tried. Newton^s Principia is not the way to 
learn calculus. Calculus can be and is presented better. There are improvements 
in ways to instruct. It is a curious fact that the way a thing is first done is usually 
an awkward way. A physicist friend gave me another example. He explained to me 
that Einstein was first led to the theory of relatr/ity by experimental facts on black 
body radiation, and then my physicist friend went on to say that, of all the ways to 
discover relativity, this is the most difficult. There were hundreds of places else- 
where where it was lying nearer the surface. It is actually true that new discoveries 
usually are awkward discoveries, and that further examination will not only simplify 
but will clarify what is actually going on. 



Another method of improvement of curriculum is by means of abstraction. Abstraction 



could be called intellectual mass production* It amounts to the following. Suppose 
you have a mathematical situation which you notice is common to several situations 
or to several problems. It may be something about the numbers which is true for 
numbers, true for complex numbers, true for vectors and true for matrices. Then 
as a matter of sheer efficiency, it is a good idea to find out what this common thing 
means once and for all and not repeat it four or five times in four or five different 
contexts. This is the meaning of mathematical abstraction, and this is the reason 
why such terms as **group»» and 'Vector space" are creeping into the high school cur- 
riculum. It is just efficiency, just mass production. 

It is also true that room is gotten for this extra achievement in mathematics by rec- 
ognizing that there are certain things, certain mathematical topics which are obso- 
lescent. I can give an example from my own experience. V/hen I studied trigonom- 
etry, I solved triangles, using logarithms and logarithms of trigonometric functions, 
and I computed by the hour every possible case for the lengths of the sides and the 
angles of the triangles. Three years later, when I had taken a B. A. , I had to actu- 
ally do some triangle solving in a surveying project. I remember my anguish when 
I fotmd out that logarithmic computation wasn't the way you did it at all. You use a 
computing machine. That had been true since long before I started high school* The 
work which I did on solution of triangles in trigonometry was a waste of my time and 
the teacher's. The advent of computing machines has made this kind of calculation 
of considerably less interest. 

There are other examples of tie same sort of thing. One that came up the other day 
is this. You remember having to learn to take square roots by a long method, tak- 
ing pairs of digits in succession? Well, that's precisely the wrong way to do it. This 
is a case of something more than obsolesc^ce. There are simpler devices for the 
computation, and there have been for at least two hundred years. 

There are other examples. In general, we must look at what we teach classically 
and revamp the curriculum, dropping out those things that are in use only because 
of tradition. This doesn't mean we change everything. We will still expect our col- 
lege freshmen to be able to solve a quadratic equation, but we expect a lot more than 
that. 

There is one last matter of change in mathematics which I want to mention. This is 
a change in what might be called clarification. One of the major achievements in 
mathematics of this century concerns the foundations of mathematics and logic. 
Let me describe tlais for you briefly, because it is not a very difficult thing, and 
it is very nice. Do you remember in Euclidian geometry that a line was defined to 
be »»that which has length but not breadth** ? Now, all of us, I think, who went 
through this were very much puzzled by this definition, and we were enormously 
relieved when we found we didn't have to use it. In general, I would like you to 
consider the following situation: Suppose you want to define every term? How 
would you go about doing this? You want to put mathematics on a really firm logi- 
cal foundation so every term you use must be defined. Very well, let's think about 
the first definition; you are defining something; you say this something is — and 
now what do you say it is, if you have defined no term? The point I am trying to 
make is that in any mathematical system there must be at least one xmdefined term. 



44 



Let's consider another example of this situation. Did you ever try to look up in a 
dictionary a word when you had no hint as to what the word mesuit? You looked up 
the word and got four or five synonyms for it. You picked the first and found the 
same word, together with the same list of synonyms. In fact, if you are com- 
pletely unfamiliar with a word and with every synonym of the word, the dictionary 
does you no good. You have to know something before you can make a definition. 

This is an informal argument. What I am trying to say is you must have some un- 
defined term in mathematics and, in particular, in Euclidian geometry. Perhaps 
you need a whole dictionary of undefined mathematical terms. This turns out not 
to be true and this is part of the achievement that I spoke of. You need just one 
undefined term. This brings us to another change which is coming into the mathe- 
matical instruction. The one notion, '^belonging to a set*^ turns out to be adequate 
for the definition of every other mathematical notion. The notion of set is natural 
and nowadays more and more mathematics is being stated in terms of sets. One 
needs the notion of set in talking about statistics^ atnd one uses the notion of set in 
plsuie geometry. A line is a set of points. I think this statement would offend no 
student of geometry, even at the very beginning of a course* 

Because of the advsuices in set theory and foundations of mathematics, we are now 
able to state more' precisely mathematical things which have been stated awkward- 
ly before. This means our curriculum must make our students aware of this more 
careful statement atnd our testing must exhibit this new clarity. 

Finally, I must mention some of the experimental curricula that are being studied. 
The School Mathematics Study Group is attempting to offer sample textbooks, and 
they have completed sample textbooks for grades seven through twelve. The Illinois 
group has completed textbooks for the ninth and tenth grades, and I think they have 
started the eleventh grade. There are many other groups. 

These changes will have an effect on testing which is probably the effect every 
change has. There is something, sometimes called the Third Law of Theory in 
Human Relationships, which is summarized: Things are going to get more mixed 
up and more difficult. 

This is true of testing. Since we are experimenting with curricula, not every high 
school will offer the same mathematics curriculum. Many students with somewhat 
different training will vrant to enter colleges, and they must be compared on their 
own merits to see who is allowed to enter. This makes a problem in testing, and 
I am glad I don^t have the responsibility for solving this problem. And this, I am 
afraid, is my sole contribution to this conference on testing. 

COLONEL O'CONNOR: Thank you. Professor Kelley. Let^s have one or two ques- 
tions for Dr. Kelley. 

MR. SAMUEL BELLMAN (Assistant Professor of English, California State Poly- 
technic College, Pomona, California); Professor Kelley, with all due respect to 
your excellent talk, why is mathematics so impenetrable to the nonmathematicail 
mind? For example, even the popular paper back, Mathematics Without Tears 
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and Neurosis , seems almost unreadable. Why are there too few people like you 
around, and what is the real reason? 

PROFESSOR KELLEY: This has worried me and every mathematician, I think, 
and I can only guess at an answer. The two disciplines in which most people have 
the most training are English and mathematics. These are the only subjects 
which are uniformly studied for at least ten years. The amount of mathematical 
background which is required of a mathematician is itself based on ten years of 
arithmetic and mathematical studies. It does seem to be the case that after ten 
years of study followed by two more high school years, followed by seven years 
of university work, the mathematical language is so complicated that it is awfully 
difficult to communicate to the nonmathematician. Part of the trouble is that the 
ability to communicate well is not necessarily concomitant with the ability to be a 
mathematician. It takes a special ability to explain something clearly. 

I don't think that the problem is entirely in mathematics* I think modern physics 
is just about as difficult to try to explain to a person who has no training. 

COLONEL O'CONNOR: One other question? 

MR. JUDSON SANDERSON (Professor of Mathematics, University of Redlands, 
Redlands, California): I am happy to be a Professor of Mathematics and I agree 
totally with what Professor Kelley has had to say. However, I would like to ask 
a question concerning testing: I try to give some modern mathematics on the grad- 
uate level, and try to give as many modern courses as possible; but then, with 
tongue in cheek, I give what they call the Graduate Record Examination created by 
the Educational Testing Service; and there are 75 questions on this examination 
and only 5 have the remotest connection to modern mathematics. When an admin- 
istrator looks at the scoi'es of this test, he thinks we don't do so well, and he 
thinks perhaps the mathematics department is rather weak. My question is: When 
are the tests going to catch up with this trend of modern mathematics? It seemed 
this test was created a long time ago. In fact, I know some of the other creators, 
and I wonder how they got their name on this test. My question is: When is the 
Educational Testing Service going to create an examination relative to modern 
mathematics? 

PROFESSOR KELLEY: Since I am not an employee of the Educational Testing Ser- 
vice, perhaps I can defend them. I see their difficulty. Modern mathematics, in 
the sense that we are using this term, is still untaught in many, many colleges. I 
think that what you say is absolutely correct, and we both know, having read fellow- 
ship applications, that one has to take test scores with certain reservations; but I 
^think that your criticism is primarily at the colleges* I suspect that the Education- 
al Testing Service will be glad to make suitable examinations as soon as we have a 
little bit more of a modern curriculum in a larger percentage of the colleges. 

COLONEL 0» CONNOR: Another question? 



MR. CURT STAFFORD (Test Officer, San Jose State College, San Jose, California): 
I think, in partial reply to this, we will get the tests from ETS. Our students are 
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probably asking us when are our tests going to resemble the courses we are teach- 
ing. My question starts off with the reverse to that story, and the fastest way to 
lose the war is to capture the opposing generals: so if we start with the premise 
we want to lose our own generals, in order to lose the war, generals--not colonels 
I left mathematics as such just about the time this stuff came in. Now, what things 
in my training should I forget, lose track of, in order to more readily grasp the 
newer concepts? What old ones should we kick out first? 

PROFESSOR KELLEY: There is very little, I suspect, in your training that we 
want to get rid of. A man who has been trained in this modern mathematics, we 
hope, knows all the important things that a man who was trained in an earlier pro- 
gram knew and, we trust, knows something else besides. 

COLONEL O'CONNOR: We wiU carry on, 
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SOME NEW SCIENCE CURRICULA AND THEIR MEASUREMENT 
FREDERICK L. FERRIS, Jr. 

Well, speaking at the tail end of a long day like this is not the most enviable spot 
to be in, I. am fully aware of the problems on the local freeways. If any of you 
do have totleaye, my feelings will not be the least bit hurt. 

However, the last time I was asked to give a talk out here in California, the con- 
ditions were much worse. The place was the University of California at Berkeley 
in connection with the new chemistry course of the Chemical Education Material 
Study, which is centered at Berkeley. Attendance was required for all teachers 
and their administrators for a new course that was being tried out in the San 
Francisco area. The day was November 8, I960, at 8:00 P. M. California time. 
This was election night. Things went off rather better than I had anticipated, how- 
ever. Chancellor Seaborg of the University of California ran the meeting. He had 
a television set brought in and at one minute before 8: OOP, M. , he declared Ken- 
nedy the winner, shut off the television, and then I was able to give my talk. Inci- 
dentally, Chancellor Seaborg is now Chairman of the United States Atomic Energy 
Commission. 

The past four years have witnessed the beginnings of what appears almost certain 
to become a veritable revolution in American education-- one leading to a genuine 
upgrading of curriculum at all levels and in all areas. 
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Looking over the first half of the Twentieth Century it is almost deplorable to note 
that curriculum development, particularly at the elementary and secondary school 
levels, but even to a considerable extent at the college level, has been largely a 
matter of voluminous compendia and lofty statements of so-called objectives, lists 
of syllabi, and the writing of textbooks in an almost whimsical or capricious man- 
ner. Often such books are fraught with error and have been written by individual 
authors who are downright incompetent in their particular field of inquiry. 

Happily, since 1956, large curriculum ventures on a scale unprecedented in Amer- 
ican history, have been evolving in rapid-fire sequence. These endeavors have 
seen some of the nation's leading scholars and scientists at the college level at long 
last concerning themselves about secondary education. The joint efforts of these 
top professionals working in close cooperation with high school teachers have al- 
ready produced vastly superior products at the high school level. The effects of 
some of these improvements are already having their impact on instruction at the 
college level. 



I am referring specifically now to five major projects, all supported by the Nation- 
al Science Foundation, and indeed there has been a real shot in the arm given to 
these projects by support since the launching of Sputnik; but it is important to re- 
member that this revolution — as I am calling it-- did start prior to Sputnik. 

Well, first of all, the first Science Study Committee, which was developed at MIT, 
really is sort of the protopype of this kind of adventure about which I am speaking 
It is under the direction of Professor Gerald Zacharias, Professor of Physical 
Science at MIT. 



The School Mathematics Study Group, which Dr. Kelley mentioned, is presently 
located at Yale University, but it seems to be moving to California, under the di- 
rection of Professor E. G. Begle. 

The Chemistry Bond Approach Project, located at Earlham College, is directed 
by Professor Lawrence E. Strong. 

The C-H-E-M Study that I mentioned, centered at the University of California at 
Berkeley, was under the chairmanship of Nobel-Prize winning Glenn T.. Seaborg, 
now Chairman of the A. E. C. 

Finally, the Biological Sciences Curriculum Study, centered at the University of 
Colorado in Boulder, was under the chairmanship of Bentley Glass, of John Hop- 
kins University. 

Now, there is little question that the scale of such ventures will steadily increase 
in the years to come. You might ask me what is the reason for such optimism. 
Well, there are several reasons. First of all, the almost fantastic growth, ac- 
ceptance, • nd use of materials that are already being prepared. Secondly, there 
is incontrovertible evidence that we have vastly underrated the capacity of United 
States high school students and presumably, therefore, those at other levels, as 
well. 
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Third, there are many similar projects already in the formative stages in other 
levels* 

Finally this is an important one, too the personal concern and vital interest 
of President Kennedy, himself, in these ventures is quite a factor. 

Now, it should be emphasized that this revolution is taking place in the American 
way. The control of the course content that is being developed in these ventures 
is solidly in the hands of the scholars and professionals. 

The use of the newly developed course materials is controlled strictly by the lo- 
cal school systems. Los Angeles, for example, is using almost all of these that 
I mentioned* 

Finally, it is important to note that only the financial support is provided by the 
Federal Government. I said »'only/* This is extremely important, but this is the 
sole role of the Federal Government here. In other words, this is a clear-cut 
example of the Federal Aid to education without control. 

I don't intend this talk in any sense to be a California for Kennedy Movement in 
1964'*. Actually, these projects started during President Eisenhower's adminis- 
tration --just about in the middle of it. 

Well, how did all this come about, this new approach to currictilum development? 
Back in 1956 there was general dissatisfaction with the deplorable state of high 
school physics instruction, not only on the part of college professors of physics 
who were deploring the entering freshmen and so on-- but there was woeful lack 
of background. When I talked to Professor Strong at Gal Tech, Chairman of the 
College Board Physics Committee, he said, *»You can assume the freshmen knew 
no physics at all a few years ago and the high school teachers, themselves, were 
bothered by the fact that the high school physics courses were simply out-of-date. 

In other words, the Nineteenth Century physics was masked by vast amounts of 
technology-- not that there is anything wrong with technology, but it isn't physics, 
as such. That's why the physicist messed Dr. Kelley's radio up. That is a tech- 
nological application of physics. Too often textbook materials have been revised 
by merely changing the picture on the cover from a DC-6 to a 707. Such '^modern 
ization** is of questionable value. 

The real credit for pioneering the formation of these study groups goes to Gerald 
Zacharias of MIT, who :*eally had the vision to see that an all-out effort or mas- 
sive attack on the problem was needed. Zacharias quickly got the financial back- 
ing of the National Science Foundation and further support, incidentally, from the 
Ford Foundation and Sloan Foundation; and this led to the formation of what be- 
came the Physical Science Study Committee. They met with some of the nation's 
top physicists, high school teachers, and other educational specialists in June of 
1957 and decided to explore what needed to be done. It wasn't long before every- 
body was in complete agreement about what needed to be done. This, in essence, 
was to start completely from scratch, and those of us who were involved in this 
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project from the beginning had no idea of what we were in for. 

The summer of 1957 saw the first steps toward the development of a brand new text- 
book, completely disregarding everything that had been taught in the past, and start- 
ing from scratch. Laboratory materials were designed to go along with this text. 
Appropriate teachers' guides, films, tests, etc. were designed to accompany the 
text. It is interesting to note in this connection, incidentally, that Zacharias also 
had the foresight to see — as far as testing is concerned that tests needed to be 
an integral part of the development of the course and not an after thought. Conse- 
quently, the development of tests designed to reflect the course objectives hadbeen 
commenced at the same time the projects, themselves, were started. This con- 
tributed markedly to the dramatic success of the Physical Science Study Commit- 
tee effort. 

One thing that struck me very recently was the fact that had this textbook been 
written five years ago by an individual author — I donH think it could have been 
no publisher would have touched it, and no teacher at the high school level would 
have dared try to use it; and yet, today we find it used all over the country. This, 
incidentally, is the first year it appeared in hard-back form. In 1957-58, it was 
used by eight schools involving 300 students. In 1958-59 this number jumped to 
286 schools, 13, 000 students. Last year, 1959-60 some 700 schools with about 
25, 000 students used it, and in the current year, 1960-61, about 1200 schools with 
nearly 50, 000 students used it. After a few years we estimate that a quarter of a 
million high school physics students will use it. Presently, something in the or- 
der of 15 per cent of the physics taught in this country utilize the PSSC materials 
and text. Soon it may run as high as 75 per cent. 

The stimulus of the Physical Science Study Committee had its carry-over into other 
fields, and projects have been launched in chemistry, biology, and mathematics. 
However, I am going to use the Physical Science Study Committee as the illustra- 
tion, a sort of prototype, both with respect to curriculum ch?»«o;es and the tech- 
niques of evaluation that we use with it. The C-H-E-M Study Group, the Biologi- 
cal Sciences Curriculum Study Group, etc. are using the same approach, but more 
is known about the Physical Science Study Committee because it has been in exist- 
ence longer. 

You might ask me at this point just how is this new course so much better than the 
so-called conventional high school physics courses of the past? In the first place, 
the authors of this book had in mind, not physics majors in college or even college- 
bound students, themselves, necessarily. This course was designed just as much 
for the terminating high school students as it was for the potential college physics 
major. 

The emphasis is away from rote memory and dishing out material in an assertive 
way. We don«t tell students that all matter is made un of molecules --memorize 
that molecules are made up of atoms — memorize that -« rather, the student is 
led to start with what he knows and discover things for himself. 

The student actually goes in with minimal equipment and measures the widths of 
the molecule, and it is startling to see how physicists have learned what they have 



50 



learned. Science is not some nice serious man in a white jacket who authoritative- 
ly tells the student some of the mysteries of the universe. 

Also, there is an effort made to get away from memorized formulae cind plugging 
in numbers end problems that are automatically worked out. This was the kind of 
physics we took in school with ten problems at the end of the chapter. That kind 
of thing ib all passe. "What is it then?'* you say to me. V/ell, the book is divided 
into four parts which include the tests and everything else. 

The first part of it was originally entitled "The Universe and Other Things". The 
Committee finally decided to be less pretentious and just call it "The Universe". 
"The Universe" was written by Professor Morrison, who is a gifted spellbinder, 
indeed, and the book reads that way. In fact, once you get into the thing, you final- 
ly find out what physics is atll about and what physics does. It concerns the mea- 
sure of time^ space, matter, compensation of time and space, leading to the no- 
tions of motion and so on. It gets into a little more chemistry, accounting for mat- 
ter, and it raises a lot of questions for students. 

Tlie second part of the book sets forth the spirit of the course. Secondly, I would 
mention that the physicists are most anxious to get across to and communicate with 
the students. There is a sort of a stop at the end of this particular view of physics. 
Now, let's take a look at something which is very obvious and see if we can study 
it. One of the most obvious things is light. You pose the question to the student, 
"What is light?" and at first this seems like a ridiculous question. Anybody knows 
what light is. However, when you stop to think of it, you really don't know very 
much about it. 

Finally, the student says, "At least we can see some of the ways in which light be- 
haves. Light casts shadows. " That is a very simple thing. Then they get into the 
fact that light can be reflected from the surfaces, and this gives relationship be- 
tween the angle of incidence and the angle of reflection. You may remember some 
of this. Then you note reflection, dispersion, and all the various phenomena — all 
the things you can observe about the way light behaves — and once you have noted 
all these various observations, there is sort of a pause about the middle of this 
part of the textbook; this is a very crucial part, too. It is pointed out to the stu- 
dents that physicists, being human like all the rest of us, given a whole set of ob- 
servations, like to put them together to formulate some kind of a model or theory 
that will e3cplain what is going on here in light; and this is the way scientific inves- 
tigation works. 

Once such a model has been developed, then it should predict other things. You 
check and see if it works. If it works, it is a good model; if it doesn't, it is not 
a good model. What is the best way of explaining how something gets from that 
light bulb to you here? You observe all these things about light, these various 
ways in which light behaves. What can be a reasonable way, a sensible way of ex- 
plaining this? Well, one of the most obvious ways is to assume the particles are 
coming from here to here. So let's follow this a littl^ bit. Let's assume light is 
just streams of particles that come out from a source.\ Let's check it now against 
all of these various observations. What about shadows? Are these particles 
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shadows ? Well, yes, streams of particles did come out and hit Wis and go around, 
and you would get a shadow. This looks pretty good for the model or the theory. 

What about reflection? If you look at balls, billiard balls, or any other kind of a 
hard surface, you notice that the angle of incidence does reach the angle of reac- 
tion , so this model begins to look better and better. Then with reflection, where 
light rays bend and so on, you can take a ball and start it rolling down an incline 
and study the incline. As it becomes steeper, the ball bends, and, if you notice, 
the relationship of reflection is wholly consistent, and so this particle model of 
light looks very good; and yet, we find that all of a sudden it breaks down indeed, 
because the model predicts that the ball, when it started to run down hill, goes 
faster when actually light, after it is bent and reflecting, slows down; so some- 
thing is wrong with the model. 

What does a physicist do when this happens? Either you'change the model or you 
start all over again with a new model. V/ell, let's try the latter. Forget about 
these particles for a minute. How else can something get from there to here? 
There is one other kind of way: take waves on water, for example. The wave it- 
self moves, but the water isn't moving in the same direction as the wave. A 
piece of cork will just pop up and down on the water as the waves pass underneath 
it. 

Well, the student works in the laboratory with a little gadget known as a Ripple 
Tank. The little vibrators go up and down, and he notices, by an intensive study 
in the laboratory of these water waves going back and forth, the behavior of light 
in the laboratory and he studies waves and finds that, sure enough, the waves re- 
flect in just the same way light does. 

Of course, shadows are caused by waves in the same way as reflection, interfer- 
ence, dispersion, and all sorts of other phenomena that art rot accounted for by 
the particle of light. Concerning this point, this little story dtops, and the stu- 
dent is left with the feeling that the wave theory seems to account for the behavior 
of light much more satisfactorily than does the particle theory. 

The course then goes on to develop force, momentu:*!, the conservation laws, elec- 
tricity, ar;d finally it leads up to modern motion, midway between the motion of 
atomic physics and some quantum mechanics. It is really up-to-date. 

I won't wAke any more time on that. I did want to give you a little picture of the 
second part of the course because it does give you an idea of what the physicists 
are trying to do here, and it also gives you the approach that is being used in these 
chemistry and biology courses that I mentioned. 

Now, how do you go about testing lor this sort of thing? This is not easy at all. It 
is easy to build the traditional multiple choice tests that you see in chemistry and 
physics. ''What is the density of aluminum?'' with five choices. This sort of thing 
is not appropriate at all to PSSC, and we had to steer a'*/ay from that. 



The approach we have used is to put the student invariably in the .ontext of a new 
situation he has never encountered before. A broad problem is given to him and 
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it is demanded that he apply what he has learned from the textbook. Now, he could 
actually use the textbook. There is no objection to bringing the textbook to the 
class and, as he is not going to find any answers to anything in the book, the new 
series of tests are going to be that way. In the test for chemistry, the students 
must bring their books to the laboratories for the test. 

Now, for the test in the laboratory. The student is presented with a slide that read^ 
•'The diagram below consists of a portion of a wave pattern generated in the Ripple 
Tank. T'le pattern is that caused by the vibration of a generator, which is not only 
vibrating, but is also moving with constcint speed in the direction indicated. " 

This is pulling a dirty trick on the student. He has this information: '*The speed 
of the generator is half as great as the speed of propagation of the waves. ^' This 
all makes sense to the student. ''The speed of the waves relative to the tank is in- 
dependent of the motion of the point source generator. *' 

This should point out to the student that actually this is a Doppler effect, but it is 
not discussed anywhere in this textbook at all, so we are trying to see if he can 
elicit the Doppler effect from this particular situation. We ask him first of all 
some rather obvious questions and we ask him some rather obvious simple ques- 
tions with relation to the wave lengths, and then something about frequency, just 
to get him thinking on the right terms, and then we pose a new one on him. To 
this time no statements have been made in the course about sound or acoustics. 
There just isn't room for it. It isn't that there is anything wrong with it this 
just makes an ideal test situation. We can tell the student that sound can be de- 
scribed as a wave motion. He is thoroughly familiar with wave motions by now, 
. and we also tell him there is a relationship between the pitch and sound and its 
frequency. 

This all again makes sense to him. Now, you asfc him tomake a prediction. This 
is the kind of thinking that the Committee is trying to elicit. "Predict what would 
happen if a body in motion was coming at you rapidly. Based on these observa- 
tions here and knowing the relationship between pitch- and frequency, the student 
may be able to deduce the idea that the pitch is going to drop because the wave 
lengths will increase £ind the frequency will decrease. 

Then we switch to light. Now, here's something that has been studied light and 
relationship of velocity, frequency, and wave lengths and so on. We pose the 
problem of a distant star that is moving toward the earth, say, at a certain speed. 
What wcild you predict about this star? What we are trying to get at here is that 
the star would appear somewhat blurred in color more than it otherwise would. 
The students would have no other basis other than his knowledge of wave motion in 
the course plus this laboratory situation upon which to proceed. These approaches 
work. This last one was used for a special College Board Test. It had to be a 
special test. A regular College Board Test wouldn't work at all on these students, 
but when this particular problem was presented to the PSSC students in the special 
College Board Test, it worked very well. 

As I said earlier, some years ago this kind of teaching was impossible at the high 
school level. Today it is being done and the students love it. 
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Now, what about the role of testing from the Physical Science Committee project? 
First, there is a series often achievement tests that are spaced throughout the 
academic year and go along with all the other course materials. These are all 
being standardized — actually they are most unique. There is nothing to compare 
the data to, because no one has ever standardized a sequence of tests like these 
before. One of the purposes of building these tests was that they serve as a frank 
empirical statement of course objectives designed to influence the way the course 
is taught. 

The physicists and the high school teachers who worked up the materials were 
afraid that teachers, once the course got out, would approach this course from a 
rote memory point of view. Therefore, they wanted to see to it that teachers had 
instruments of the sort I showed you here, which measure the kinds of deduction, 
productive thinking, etc., that were expected of students taking this course. Many 
times the question is raised, '^Should tests influence curriculum? »» Here is a case 
^vhere we deliberately set out to do it. 

A second purpose of testing in the context of the Physical Science Study Committee 
project was that of feedback and evaluation. Here you can^t use a conventional de- 
sign you might be familiar with from courses and testing measurements and so on. 
Namely, let's get an experimental group and a control group, and they have some 
criteria measurements here, from analyses of other tests. That won't work here 
at all because there is no criterion measurement. Inere are criteria measure- 
ments designed by the Physical Science Study Committee, but they won't work on 
your control group at all and vice versa. 

Well, how do you go about it? In the first place, the question is not one of wheth- 
er this physics is good or bad. Physics is good, I submit. Who is to say what is 
good physics if it isn't the nation's leading physicists, and we have had them work- 
ing on this projv ct in abundance. Whether it is good or not is not the question. The 
question is: this material teachable at the level and for the group for which it 
was designed? 

Most high school physics students lie above the 75 percentile in respect to scho- 
lastic ability, and the approach we used in the studies designed for the PSSC. 
Other curriculum studies, by the way, are giving a scholastic aptitude test to all 
students, and this has been done for several years in a row now. We have ac- 
quired much data, and this battery of ten achievement tests has been helpfuL 

Let's say this is an advanced placement course, with the upper 5 per cent the 
very brightest of students and this is one of the things we wanted to see. Is 
this so or not? V/hat about the validity of the tests? Well, here we have a very 
peculiar kind of vaJidity, I think. It is sort of a content validity. I think it is 
more than that. It is possibly a validity by definition. All of these test terms 
were gone over by all of the physicists on the project, and we never used one of 
these tests until everyone on the project was happy with them. 

In other words, by definition, these tests are representative of the kinds of things 
the Committee — the textbook authors had in mind. If the students can handle 
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thi^ kind of material, we will be very happy-- delighted, in fact. 

As far as validity coefficients go, we could get beautiful coefficients with grades, 
because eill the teachers from almost all the schools use test scores as part of 
their grades. It is obvious that we haven't even. bothered to try it. 

As far as the tests are concerned, they turned out to be of middle difficulty. This 
surprised us. On the first one all the physicists at MIT though^ the tests were 
going to be much too difficult. All of us were astonished as to how well the stu- 
dents did on these tests. One of the most encouraging things the first year of the 
project was that nobody believed this would happen, considering the level of ap- 
parent difficulty. That's why I said earlier that we have simply underestimated 
what the students can do when they are motivated and interested. 

These are forty- five minute tests, and their reliability runs around . 75; a com- 
posite reliability of all the tests treated as a single test was . 95 plus. Now, one 
of the amazing facts that has resulted and one of the things I wart to show you be- 
fore concluding-- if I can take a few minutes is the inordinately low correlation . 
of achievement with scholastic aptitude. This is an astonishing fact. 

I have been working with Lee Cronback from Illinois and Jerome Bruner from 
Harvard on some of these studies. We still are puzzled by this. There is another 
very peculiar thing that happens. (Consult Table 2)* If you take these tests in se- 
quence, the first PSSC Tests correlate around . 50 with SCAT, by midyear .40, 
and by the end of the year the correlation has slipped down to around . 20. There 
is a significant percentage of students who are dubbed "low aptitude" by aptitude 
tests. I want to emphasize that point. The aptitude tests say they are low aptitude 
or their grades and other subjects even may say they are of low aptitude, but actu- 
ally these are students who are bored and refuse and rebel at the idea of rote mem- 
ory. They refuse to sit down and memorize all the definitions, to cram the night 
before the exam, to burn the midnight oil. All these indications are indicative of 
American education in termp of interest. Where one student might say, "Once I 
get excited by such things as measuring the thicknesses of the molecule. . . . ", they 
find there is really something here, and you will find that the correlation is in- 
creasingly depressed by the fact that these "low aptitude" students are penetrating 
the higher ranges of achievement, and the opposite works, too. 

Many bright students, who bank on this ability to cram, just won't work in this 
course at all, and they won't do well. I have a series of slides I want to run 
through in rapid-fire sequence, and I want you to see the change from "low ap- 
titude" and what happens as the first year progresses. 

The first slide (Table 1) shows the aptitude of the selected group of students. These 
norms are drawn from the 1958- 59 and the 1959 - 60 academic years. You will 
note that we are dealing with essentially the upper 75 per cent of the United States 
twelfth grade students. I had better explain the slide because all of the remaining 
slides will be similar to this one. 



* All slides shown in Dr. Ferris' talk are reproduced in table form following page 56. 



55 



The fat, shaded part of the graph represents 50 per cent of the population. The 
upper limit of the shaded bar represents the 75th percentile while the lower end 
of this shaded bar represents the 25th percentile. In each case the graph repre- 
sents only 90 per cent of the group in question; consequently, the lower end of the 
vertical line represents the 5th percentile while the upper end of the vertical line 
represents the 95th percentile. Median SCAT scores are represented by the short 
horizontal line across the shaded bar of each graph. The dotted horizontal line at 
a SCAT score of 302 represents a score of 500 on the CEEB - SAT. 

By comparing the graph for PSSC students' SCAT scores with that of the United 
States twelfth grade students, you can readily see that we are dealing with an ex- 
traordinarily able group of individuals. You will note that the lower quartUe for 
the PSSC students falls at approximately the 80th percentile when compared to the 
United States twelfth grade students, Y/hile the selected PSSC students have 
fair number of individuals who dip into the lower aptitude range, essentially we 
are dealing with the upper 20 per cent of United States twelfth grade students. 

The next sequence of slides (Tables 3-11) will show the relationship of PSSC 
Achievement Test scores by aptitude groups for the various tests during the course. 
The PSSC students are broken up into three aptitude groups -- above the 90th per- 
centile, 90th to 75th percentile, and 75th to 60th percentile -- because we are in- 
terested in seeing whether or not the course is appropriate only for the brightest 
students. 

On the extreme left of each table, scores from 40 to 220 represent the standard 
scores on the Achievement Tests of the PSSC. The solid horizontal line at 130 
represents the mean, and the dotted lines at 90 and 170 represent the standard 
deviation of plus and minus 40. Having placed the Achievement scores on such a 
scale, we are able to compare the sequence of aptitude with achievement, since 
differences and difficulty are eliminated through the use of standard scores. 

As we examine these tables, it will be well to keep in mind that the 90th to 75th 
percentile repres it the lower limit of the group for which the course was de- 
signed. The third column, that is, the aptitude range of 75th to 60th percentile, 
represents those students whose scholastic aptitude, as measured by scholastic 
aptitude tests, are below the level for which the course was designed. 

Now, (refer to Table 3) the very first test on the first half of Volume I demon- 
strates one important thing. Here is a rather marked overlap among these groups 
with a fair percentage of students in the middle aptitude range doing better than 
the median of the brightest students, and likewise some of the students in the low- 
er aptitude level are performing at about the median of the brightest students. 

We are going to proceed through the remaining slides in rapid sequence. (Follow 
Tables in back) I want you to see what happens to this group (that is, the 75th to 
60th percentile SCAT group) with respect to the total group as it progresses 
through the PSSC course and the students get into more and more complicated 
physics. (Shown in Table 6) 
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In the second half of Volume II the students are becoming exposed to very compli- 
cated modern quantum mechanics. The first half of Volume II was describing ob- 
jects, if you will remember, and this, the second part, deals with more compli- 
cated physics. Notice the overlap of the 75th to 60th percentile group with the to- 
tal group. As we proceed from the first three tables to Table 6 (which is the se- 
cond half of Volume II) we note that the overlap is getting greater and greater. The 
next slide (Table 7) is a midyear comprehensive examination on the full first half- 
year's work. 

The next slide (Table 8) is the first test covering Volume IIL Here, the students 
are getting into dynamics, forehand momentum laws. The next slide (Table 9)is 
the last half of Volume IIL This slide is an extraordinary one. Look at the per- 
formance of the number of students above the median of the very brightest stu- 
dents. Correlation at this point between SCAT and PSSC Achievement, as I recall, 
is about . 28 (consult Table 2), The last slide shows a composite test on the second 
semester's work (Table 11). 

The number of cases here wasn't terribly large, and we began to wonder about the 
significance of this. I'd like to conclude with a couple of bothersome questions 
here. When students perform at the 90th percentile on these tests of modern 
physics they are certainly not dumb, and they obviously are not low aptitude stu- 
dents. Anybody who can handle these - -you just take a look at these tests any- 
body who performs at 90th percentile there has something on the ball. 

Now, if our hypothesis concerning the interpretation of these data is correct, I'd 
like to raise the question, ^'Does this pose a question about the nature of existing 
scholastic aptitude tests and those high correlates with them, namely 'grades' in 
school and college?" 

Scholastic aptitude tests predict college grades well, so we select on the basis of 
the tests and grades in the same old way. If our hypothesis is correct, somehow 
or other we have to break out of this pattern and really do what we say we are 
going to do in teaching and grading students. If we do not, we pay mere lip ser- 
vice to lofty objectives and reward students on their ability to cram what is in the 
book and spew it out on the day of the final examination. I personally believe that 
the experience of the Physical Science Study Committee and the experience of the 
other curricular projects have shed some light on these matters and should help 
us to go a long way in producing tests that are both more dynamic and appropriate 
for modern education. 
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COLONEL O'CONNOR: That was very fine, Fred, We won't take time for ques- 
tions, but there may be individuals who want to come up and ask questions at the 
table, 

I do want to thank those two gentlemen. They gave two magnificent talks this 
afternoon, and the enthusiasm of Fred Ferris is typical of the enthusiasm of the 
whole Physical Science Study Committee since its inception. 

Finally, I want to express your appreciation and my appreciation to John Helmick 
for having us here. I thank you for coming here and being so patient with us, 

(The meeting was adjourned at 3:35 P. M. ) 
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Explanatory Notes: 

1. Percentiles indicated are based on national norms 
group . 

2, In each case, graphs represent 90% of ^e group in 
question; 50% of each group represented by broad 
segment of each graph. 



3, Median scores represented by short horlzoital line 
across each graph. 
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Table of Correlations with SCAT 

r 

Test on Volume I (Chapters 1-6) 51 

Test on Volume I (Chapters 7-10) 54 

Test on Volume II (Chapters 1-5) 41 

Test on Volume II (Chapters 6-9) 45 

Test on Volumes I and II 40 

Test on Volume III (Chapters 1-4) 44 

Test on Volume III (Chapters 5-7) 37 

Test on Volume IV (Chapters 1-5) 24 



Table of Correlations with SAT Verbal 

CEEB-PSSC Physics .45 

CEEB Regular Physics .61 
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Tables 3-11 



Explanatory Notes: 

1. Percentile ranges Indicated are based on the national 
norms group . 

2. In every case, the graphs represented 90% of the PSSC 
students falling in each category; 50% of the students 
are represented by the broad segment of the graph. 

3. Median scores represented by short horizontal line 
across each graph. 

4. The scale used for performance on the achievement 
tests is an arbitrary standard scale , with a mean of 
130 and a standard deviation of 40 in every case. 
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PSSC ACHIEVEMENT TEST SCORES BY APTITUDE GROUPS 
Test on Volume II (Chapters 1-5) 
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Test on Volume III (Chapters 1-4) 
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PSSC ACHIEVEMENT TEST SCORES BY APTITUDE GROUPS 
Test on Volume IV (Chapters 1 - 5) 
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PSSC ACHIEVEMENT TEST SCORES BY APTITUDE GROUPS 
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Guidance Coordinator, Burbank Ifigh School, Burbank^ 
California 

Counselor, Los Angeles State CoUege, Los Angeles, 
California 

Director of Special Services, Enterprise School Dis- 
trict^ Compton, California 

Supervisor of Guidance, Valley West Elementary 
District, Los Angeles City Schools, Reseda, Cali- 
fornia 

Coordinator of Secondary Curriculum, Pomona Um* 
fied Schools, Pomona, California 

Psychologist, Fullerton High Schools and Junior Col- 
lege Districts, Fullerton, California 
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S. Helen Brown 
Erven Brundage 

Carl J. Burk 

Robert K. Burley 

Tom Butterworth 

John Ccilkins 
William C. Calvert 

Dorothy N. Candland 

Lelon Capp8 

CeciUe Carnes 

Michael Cassidy 
Margaret L, Chadwick 

Helen E. Chapin 

James Chapman 

JohnH. Chilcott 

Inge Chwang 

Corabelle Clark 



Chief of Guidance, Santa Barbara County Schools, 
Goleta, California 

Director, Pupil Persoimel Service, Department of 
Education, San Diego County Schools, San Diego, Cal- 
ifornia 

Assistant Superintendent, Instruction, Compton Union 
High School District, Compton, California 

Coordinator of Guidance, Redlands School District, Red- 
lands, California 

Consultant, Research and Guidance , Los Angeles County 
Schools, Los Angeles, California 

Principal, Loma Vista School, Ventura, California 

Head, Psychology Department, La Verne College, La 
Verne, California 

Director of Curriculum, Covina- Valley Unified School 
District, Covina, California 

Assistant Professor of Education, University of Cali- 
fornia, Santa Barbara, University, California 

Teacher, English and Mathematics, Marlborough 
School, Los Angeles, California 

Principal, Hollydale School, Hollydale, California 

Education Director, Chadwick School, Rolling Hills, 
California 

Curriculum Coordinator, Burbank High School, Bur- 
bank, California 

Head, Mathematics and Engineering Department, An- 
telope Valley College, Lancaster, California 

Assistant Professor of Education, University of Cal- 
ifornia, Santa Barbara, University, California 

Assistant to Coordinator of Testing Services, Educa* 
tional Testing Service, Los Angeles, California 

Director of Guidance, Compton City Schools, Comp- 
ton, California • 
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Edwin C. Clark 

Marilyn Clark 

Pearl E. Clark 

V/illiam E. Clarke 

William Coleman 

Marie Connally 

Robert V/. Coombs 

Dorothy Craig 
Max L. Crawford 

Howard M. CritcheU 

Margaret Crowley 

Margaret Davison 

Al S. Derian 

Marvin N. Dillon 
Elizabeth Dimock 

Valeria Dotterrer 

Glenn W, DurfUnger 

Floyd Eastwood 
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Director of Research, Burbank Unified Schools, Bur- 
bank, California 

Coordinator, Testing Services, Educational Testing 
Service, Los Angeles, California 

Dean of Students, Chaffey College, Alta Loma, Cali- 
fornia 

Guidance Coordinator, San Diego County Schools, San 
Diego, California 

Consultant, Houghton Mifflin Company, Santa Monica, 
California 

Counselor, JVIark Keppel High School, Alhambra, Cal- 
ifornia 

Director of Special Services, Ventura City Elementary 
School District, Ventura, California 

Teacher, Marlborough School, Los Angeles, California 

Coordinator of Guidance, John Burroughs High School, 
Burbank, California 

Director, Bellet Foundation, Palos Verdes Estates, Cal- 
ifornia 

Supervisor of Guidance, West Elementary District, Los 
Angeles County Schools, Los Angeles, California 

Assistant Superintendent, Educational Services, Comp- 
ton City Schools, Compton, California 

Counselor - Psychometrist, Long Beach City College, 
Long Beach, California 

Test Officer, Chico State College, Chico, CaUfornia 

Vice Principal, San Bernardino High School, San Ber- 
nardino, California 

Counselor, East Los Angeles College, Los Aceles, 
California 

Chairman, Department of Education, University of 
California, Santa Barbara, University, CaUfornia 

Counselor, Los Angeles State College, Los Angeles, 
California 



Raymond Eddy 

Raymond C. Ede 

Robert L. Egbert 

Celia Einbinder 

Merle H. Elliott 

Mrs. R. W. Elliott 
Betty Ellis 

Gladys Engelbrecht 

Darrell Finnegan, S#J. 

J. Donald Fisher 

Thomas J. Fitzgibbon 

Evelyn Foley 

Harry J. Ford 

Richard Fortna 

Stanley D. Frank 

Alvin Frcy 

Jeannette E. Friedrich 
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Consultant, California Test Bureau, Sherman Oaks, 
California 

Headmaster, Army - Navy Academy, Carlsbad, Cali- 
fornia 

Professor of Educational Psychology, Brigham Young 
University, Provo, Utah 

Secretary, Advisory Services, Educational Testing Ser- 
vice, Los Angeles, California 

Director of Research, Oakland Public Schools, Oakland, 
CaULifornia 

Director, Desert Sun School, Idyllwild, California 

Supervisor, Evaluation and Research, Los Angeles 
City Schools, Los Angeles, California 

Coiinselor, Los Angeles State College, Los Angeles, 
California 

Director, Guidance Center, Loyola University, Los 
Angeles, California 

Head Counselor, Crescenta Valley High School, La 
Crescent a, California 

Director of Field Services, California Test Bureau, 
Monterey, California 

Head, EngUsh Department, Antelope Valley College, 
Lancaster, California 

Curriculum Consultant, Covina - Valley Unified School 
District, Covina, California 

Associate, Advisory Services, Educational Testing 
Service, Los Angeles, California 

Guidance Coordinator, San Diego County Schools, San 
Diego, California 

Psychologist, SantE: Barbara County Schools, Goleta, 
Calif orziia 

Consultant in Elementary Research and Guidance, Los 
Angeles County Schools, Los Angeles, California 
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James R. Froyd 
Gordon Gardner 
Jerry C. Oarlock 
James D, Gilmour 
Clayton M« Gjerde 
Lucy Goldberg 
Robert €• Goodridge 
Garford G, Gordon 
Leonard V» Gordon 
Bette M» Gordy 
Janet Gottfredson 
Merrill V* Goudie 
John C. Gowan 

J* Rollin Grant 
Betty Grauer 
John Greene 

O 

ERLC 



Senior Psychologist, Advisement Service, Los Angeles 
City Schools, Los Angeles, California 

Director of Curriculum, Riverside County Schools, Riv- 
erside, California 

Consultant in Educational Research, Los Angeles County 
Schools, Los Angeles, California 

Director, Student Personnel Services, Occidental Col- 
lege, Los Angeles, Catlifornia 

Coordinator of Graduate Studies, San Diego State College^ 
San Diego, California 

Admissions Office, University of California, Los Ange- 
les, California 

Dean of Students, University of Redlands, Redlands, Cal- 
ifornia 

Research Executive, California Teachers Association, 
Burlingame, California 

Department Director, U, S, Naval Persoimel Research 
Field Activity, San Diego, California 

Principal Clerk, Admissions Of^ce, University of Cal- 
ifornia, Los Angeles, California 

Psychologist, Glendale Unified School District, Glen- 
dale, California 

Director, Division of Secondary Education, Los Ange- 
les County Schools, Los Angeles, California 

Associate Professor of Education, Chairman of Guid- 
ance Area, San Fernando Valley State College, North- 
ridge, California 

Headmaster, Francis Parker School, San Diego, 
CaUfomia 

Head, Scoring and Clerical Services, Educational Test-* 
ing Service, Los Angeles, California 

Counselor, Santa Barbara High School, Santa Barbara, 
California 
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Milton Grossman 

Helen C. Gunter 

Howard R. Haering 

Richard W. Hamilton 

Dwight Hanawalt 

Mrs. Donald Hanson 
Joseph T. Hanson 

Wesley L. Harmon 

James M. Harris 

Lucille Harris 

Jessica Haskell 

R* Garn Haycock 

Donald Hays 

Allen Hearne 
Harrison F, Heath 
John Helmick 
Charles Herbert 
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School Psychologist, Sweetwater Union High School Dis- 
trict, Chula Vista, California 

Curriculum Coordinator (Research), San Mateo City 
Schools, San Mateo, California 

Consultant in Guidance, Psychologist, San Bernardino 
County Schools, San Bernardino, California 

Psychometrist, San Fernando Valley State College, 
Northridge, California 

Dean of Men, Head of Physical Education Department, 
La Verne College, La Verne, California 

Teacher, Sequoyah School, Pasadena, California 

Director of Research, California Teachers Associ- 
ation, Southern Section, Los Angeles, California 

Associate Dean of Guidance, Chaffey College, Alta 
Loma, California 

Associate Professor of Educational Psychology, Brig- 
ham Young University, Provo, Utah 

Supervisor, Test-Scoring Services, Orange County 
Schools, Santa Ana, California 

Assistant Principal and Director of Guidance, Pomona 
High School, Pomona, California 

Vice Principal for Guidance, Redlands High School, 
Redlands, California 

Director, Pupil Personnel Services, FuUerton Union 
High School and Junior College District, Fuller ton, 
California 

Supervisor, Testing and Research, Richmond Schools, 
Richmond, California 

Test Officer, San Jose State College, San Jose, Cali- 
fornia 

Director, Educational Testing Service, Los Angeles, 
Califorziia 

Coordinator, Secondary Foreign Language and Eng- 
lish, San Bernardino County Schools, San Bernardino, 
California 
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William C* Higman 
Elise Hobden 
Byron M, Holmes 
Ralph Hone 
Fred Hooper 
Robert E. Horton 
Robert Howe 
Howard Hubbard 

Clyde Jack 
Janet Jacobs 

Iva Jex 

Boyd A. Johnson 
Hazel M» Johnson 
Berdine J. Jones 
Sam Kavruck 
Phyllis Kennedy 
Katherine Killeen 
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Assistant Superintendent in Charge of Instruction, San 
Marino Unified School District, San Marino, California 

Counselor, Admissions Office, University of California, 
Los Angeles, California 

Assistant Dean, Los Angeles City College, Los Angeles, 
California 

Professor of English, University of Redlands, Redlands, 
California 

Director of Studies, Webb School of California, Clare- 
mont , California 

Assistant Dean of Instruction, Los Angeles City College, 
Los Angelest California 

Assistsmt in Guidance and Research, San Bernardino 
City Schools, San Bernardino, California 

Supervisor* High School Education (Science and Mathe- 
matics), Long Beach Unified Schools, Long Beach, Cal* 
ifornia 

Counselor* Cboffey College, Alta Loma, California 

Psychometrist, La Sierra College, La Sierra, Cali*- 
fornia 

Manager, School Testing Service, Berkeley, Csdifornia 

Counselor* Redlands Junior High School, Redlands, Cal- 
ifornia 

Counselor, Los Angeles State College, Los Angeles, Cal* 
ifornia 

Consultant* Research and Guidance, Los Angeles County 
Schools* Los Angeles, California 

Specialist* Testing Programs, United States Office of 
Bducation, Washington, C* 

Testing Supervisor, San Fernando Valley State College, 
Northridge* California 

Counselor, Buena Park High School, Buena Park, Cal- 
ifornia 



Margaret Kindred 
Jamee E. Klabn 
Robert R. Kaapp 
George A, Koester 
Theodore E, Kopp 
Reinhold Krantz 
David P, Kressen 
Nadine M. Lambert 

Robert Lambert 

Vernon F. Lars en 

Alexander L Law 

Deloris Leander 

Charles R. Lewin 

Clark Lewis 
Hazel Lewis 

Ruasel L, Lewis 

Charles S. Locks 
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District Superintendent and Principal, Hope School, 
Santa Barbara, California 

Coordinator, Psychological Services, Tacoma Public 
Schools, Tacoma, Washington 

Project Director, S. Naval Personnel Research 
Field Activity, San Diego, Califorxxia 

Chairman, Department of Education, San Diego State 
College, San Diego, California 

Retired, Burbank Senior High School, Burbank, Cal- 
ifornia 

Professor of Chemistry, University of Redlands, Red- 
lands, California ' 

Assistant Headmaster, Polytechnic School, Pasadena, 
California. 

Research Department, State Project « Emotionally 
Disturbed Children, Califorxua StatjB Department of 
Education, Los Angeles, California 

Assistant to the Director, Advisory Services, Educa- 
tional Testing Service, Los Angeles, California 

Western ^egioa Director, Science Research Associ- 
ates, Oakland, California 

Associate, Advisory Services, Educational Testing 
Service, Los Angeles, California 

Administrative A8sis^;ant, Educational Testing Ser- 
vice, Los Angeles, California 

Assistant Principal, Inglewood High School, Ingle- 
wood, California 

Principal, DeAnza School, Ontario, California 

Administrate a Assistant - Research, Stockton 
Unified '^ichools^ Stockton, CaliforrJa 

Dean of Admissions and Guidance, Santa Monica 
City College, Santa Monica, California 

Instructor in Psychology, Los Angeles Valley Col- 
lege, Van Nuys, California 
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Marguerite Long 
Robert Lorentz 
Lawrence A. Lucas 
Orral S. Luke 
Marguret S» Lund 

Carol Lundberg 

Arthur Lynn 

Helen Mc Cannon 
Clyde C. McGully 

J. W. McDaniel . 

Robert W, McQueen 

Edwin MacDonald 

T. Leslie MacMitchell 

Fred Machetanz 

James Magary - 

Alberta R. Magaw 

Ft Grant Marsh 
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Registrar and Counselor, Marlborough School, Los 
Angel '3s, California 

Consultant, Secondary Edifcation, Los Angeles 
County Schools,. Los Angeles, California 

Director, Educational Services, Whittier Union High 
School Districti Whittier, California 

Director, Guidance and Testing, Bakersfield College, 
Baker sfi eld, California 

Coordinator of Guidance and Special Education, Man- 
hattan Beach City Schools, Manhattan Beach, Cali- 
fornia 

Psychologist, Enterprise School District, Compton, 
California 

Dean of Guidance, Fullerton Union High School, Ful- 
lerton, California 

Counselor, Azusa High School, Azusa, California 

Dean of Instruction, Antelope Valley College, Lan- 
caster, California 

Vice President, San Bernardino Valley College, San 
Bernardino, California 

Counselor, Emerson Junior High School, Pomona, 
California 

Counselor. Los Angeles Valley College, Van Nuys, 
Califom: ^ 

Director, College Entrance Examination Board West- 
/ern Regional Office, Palo Alto, California 

Coordinator of Testing, Los Angeles Valley College, 
Van Nuy8» California 

Professor of Education, University of Southern Cali- 
fornia, Los Angeles, California^ 

Coordinator, Tests and Guidance, Montebello Senior 
High School, Montebello, California 

Coordinator, Testing Services, San Francisco State 
College, San Francisco, California 
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Jack Matlock 
James Mead 
Spencer L. Menzel 
Sister Mercia Louise 
Tomi Miyata 
James Moffat 
Ivy Mooring 
Mary Etta Murray 
Else Nelson 
Winston Nelson 
Thomas Nesbitt 
Esther G. Nolan 
W. L. Nuckolls 
Arden A, O'Brien 
Vane dinger 
Margaret L. Orear 
Mary Owen 
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Consultant, California Test Bureau, Long Beach, Cal- 
ifornia 

Psychometrist, Baldwin Park Unified Schools, Bald- 
win Park, California 

Coordinator of Curriculum, Sweetwater Union High 
School District, Chula Vista, California 

Dean of Women, Mount St, Mary's College, Los An- 
geles, California 

Graphics Clerk, General Services, Educational Test- 
ing Service, Los Angeles, California 

Counselor, La Habra High School, FuUerton, Cali- 
fornia 

Direct9r of Research and Guidance, Palos Verdes 
School District, Rolling Hills, California 

Associate Dean of Students, California State Poly- 
technic College, San Luis Obispo, California 

Department of Education, Pacific Union Conference, 
Glendale, C alif o r ni a 

Assistant Superintendent, Educational Services, Po<* 
mona Unified Schools, Pomona, California 

Director of Testing, Research and Guidance, City 
College of San Francisco, San Francisco, California 

Consultatnt in Research and Guidance, Los Angeles 
County Schools, Los Angeles, California 

Administrative Trainee, Ventura City Elementary 
Schools, Ventura, California 

Associate, Advisory Services, Educational Testing 
Service, Los Angeles, California 

Coordinator, Testing and Guidance, Sweetwater 
Union High School t^'-strict, Chula Vista, California 

Deputy Superintendent, Bellflower Unified Schools, 
Bellflower, California 

Administrative Assistant, Advisory Services, Edu- 
cational Testing Service, Los Angeles, California 
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Robert K. Oyler 

Dale C. Paul 

Warren Peace 
Douglas B. Peatrce 

Raymond C, Perry- 
Vernon R, Persd^. 
John Pettingall 
Glen N. Herson 
Raymond J. Pitts 
Leonard Porterfleld 
Mabel C. Purl 

Gladys W. Quinlin 
Harold J« Reed 

Henry Reed 

David Re..^ j 

Irvin F. Richardson 

Sybil K. Richardson 



Director of Guidance, Claremont Unified Schools, Clare- 
monti California 

Prixicipal, Poinsettia Elementary School, Ventura, Cal- 
ifornia 

Counselor, Pioneer High School, V/hittier, California 

CounsAlor, East Los Angeles Junior College, Los An- 
g ele s, Calif o rnj. a 

Professor of Education, University of Southern Cali- 
fornia, Los Angeles, California 

Manager, Counseling Center, Uzdversity of California, 
Santa Barbara, University, California 

Cooxdinator of Gifted Program, San Gabriel Schools, 
San Gabriel, California 

Guidance Coordinator, San Diego County Schools, San 
Diego, Califorzda 

Associate Professor, Education Department, Los An- 
geles State College, Los Angeles, California 

Dean of Guidance, Allan Hancock College, Santa Maria, 
California 

Director of Research and Testing, Riverside City 
Schools, Riverside, Califorzda 

Principal, Chadwick School, Rolling Hills, California 

ConsultcUit, Division of Research and Guidance, Los 
Angeles County Schools, Los Angeles, California 

Guidance Coordinator* San Diego Cotmty Schools, San 
Diego, California 

Assistant Principal, Morningside High School, Ingle- 
wood, California 

Counselor, East Los Angeles Junior College, Los An-* 
geles, California 

Associate Professor, San Fernando Valley State Col- 
lege, Northridge, California 
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William V. Rickel 
John Joe Risser 
Charles F. Robar 
Mildred C. Robeck 
Thomas B. Rodger s 
Shirley Root 
Paul Rosendahl 
Dorothy Roy 
Howard L. Rupard 
Juan Salinda 
Katharine A. Salter 
Jack Salyers 
Marie Sander 
Judson Sanderson 
Elnora Schmadel 
William Schoenbach 
Eugene Schumacher 
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Coordinator of Testing, Anaheim Union High School 
District, Anaheim, California 

Psychologist, Pasadena City College, Pasadena, Cal- 
ifornia 

Director of Personnel, Ontario School District, On- 
tario, California 

Assistant Professor of Education, University of Cal- 
ifornia, Santa Barbara, University, California 

Assistant Director, Educational Testing Service, Los 
Angeles, California 

Counselor, E. M. Cope Junior High School, Redlands, 
California 

Director of Giddance, Santa Maria High School, Santa 
Maria, California 

Executive Secretary, Educational Testing Service, Los 
Angeles, California 

Director of Testing and Research, 'Montebello Senior 
High School, Montebello, California 

Staff Psychometrist, Loyola University, Los An ^eles, 
California 

Assistant Director, Associational Activities, College 
Entrance Examination Board, New York, New York 

Superintendent, El Monte Union High School District, 
El Monte, California 

Coordinator of Guidance, Bellflower Unlived Schools, 
BellHower, California 

Professor of Mathematics, University of RedlandSi 
Redlands, California 

Director, Research and Guidance, San Gabriel School 
District, San Gabriel, California 

Principal, Lincoln and Pierpont Schools, Ventura, 
California 

Assistant Dean of Students, Antelope Valley College, 
Lancaster, California 
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Paul Schumann 

Jim Schunert 

Harlan E. Selvey 

Carleton B« Shay 

Ethel F* Sheldon 
Thomas Shellhammer 

Patricia Simmons 

WiUiamM. Slaichert 

Merl F. Sloan 

George Smith 

W. Holmes Smith 

Dorothy Soeberg 

Madeline C« Sorensen 

Gavth Sorenson 

Leroy Spore 

Curt Stafford 

Edward Stancik 
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Counselor^ Pioneer High School, Whittier, Caiifornic. 

Professor of Education, San Diego State Coliege, San 
Diego, California 

Registrar, Southern California College, Costa Mesa, 
California 

Director of Testing, Sa \. * Monica High School, Santa 
Monica, California 

Director, The Sequoyah School, Pasadena, California 

Consultant, Bureau of Education Research, California 
State Department of Education, Sacramento, California 

Director of Research, Orange County Schools, Santa Ana, 
California 

Associate Professor of Education, Denver University, 
Denver, Colorado 

Director of Student Personnel, El Camino College, £1 
Camino College, California 

Intern, Pasadena Counseling Center, Pasadena, Cali- 
fornia 

Assistant Director of Student Personnel, El Camino 
College^ El Camino College, California 

Assistant Professor of Education, Los Angeles State 
College, Lios Angeles, California 

Assistant' Principal, Pioneer High School, Whittier, 
California 

Associate Professor of Education, University of Cal** 
ifornia, Los Angeles, California 

Curriculum Coordinator, Tulare Coxmty Schools, Vi- 
salia, California 

Test Officer, San Jose State College, San Jose, Cal* 
ifornia 

Staff Associate, Science Research Associates, Orange, 
California 



Leslie H. Standlea 

Doyle Stansel 

Alfreda A. Stapinski 

John H, Steinemann 

John S. Stevens 

Mildred Stevens 

Cecil C. Stewart 

Sterling Stott 

Kathleen Strickland 

Paul M* Summit 

Philip Sutherland 

Marvin R. Tate 
Ray L» Thomas 

Anton Thompson ^ 

Fred W. Thompson. 

Muriel Thomson 

Louis P. Thorpe 
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Assistamt Dean of Students, Chaffey College, Alta 
Loma, California 

Test Officer, California State Polytechnic College, 
Pomona, California 

District Psychologist, San Marino Schools, San Mar* 
ino, California 

Project Director, Uaxted States Naval* Personnel Re- 
search Field Activity, San Diego, California 

Dean of Counseling and Guidance, Mt. San Antonio 
College, Walnut, California 

Class Counselor, Redlands Senior High School^ Red- 
lands, California 

Head, Guidance and Pupil Pi&rsonnel Department, Los 
Angeles State College, Los Angeles, California 

Guidance Supervisor, University Elementary School, 
University of California, Los Angeles, California 

Counselor, Alhambra High School, Alhambra, Cali- 
fornia 

Coordinator of Special Services, Palm Springs Uni- 
fied Schools, Palm Springs, California 

Psychometrist, Hudson School District, La Puente, 
California 

Principal, Edison School, Ontario, California 

Assistant Principal, Ganesha Highr School, Pomona, 
California 

Director of Research, Long Beach Unified School 8» 
Long Beach, California 

Dean of Counseling, San Fernando Valley State Col- 
lege^ San Fernando, California 

Consultant, Division oi Research and Guidance, Los 
Angeles County Schools, Los Angeles, California 

Professor of Education, University of Southern Cali- 
fornia, Los Angeles, California 
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Arnold Tilden 

Sarah Tilden 

Arthur L, Tollefson 

Dorothy Tucker 

David Uslan 

E. L. Van Dalsem 

Nancy Venuto 

Frances Ver Bryck 

Hazel Vickers 

Vet a B. Vroman 

Burton W, Wad^worth 

Gunnar Wahlquist 

Betty Walthall 

Rosalie Waltz 

Edwin Wandt 

Florence E. Ware 

HexiryL Weitzel 



Dean of Liberal Arts College, Arizona State Uni- 
versity, Tempe, Arizona 

School Social Worker, Phoenix School District No, 1, 
Phoenix, Arizona 

Manager, Pasadena Counseling Center, Pasadena, 
California 

Associate Professor of Psychology, California State 
Polytechnic College, Pomona, California 

Principal, El Canriino Elementary School, Ventura, 
California 

Director, Guidance and Research, Sequoia Union 
High School District, Redwood City, California 

Curriculiun Consultant, Covina- Valley Unified 
School District, Covina, California 

Counselor, Los Angeles Harbor College, Wilming- 
ton, Califoraiia 

Assistant Principaly Fremont Junior High School, 
—Pomona, California 

Director of Special Services, Pomona Unified Schools, 
Pomona, California 

Dean of Student St Antelope Valley College, Lancas- 
ter, California 

Assistant Superintendent, Instruction, El Monte 
Union High School District, El Monta, California 

Psychometrist, Ranchito School District^ Pico- 
Rivera, California 

Director of Guidance Services, Santa Monisa Uni* 
fled School District, Santa Monica, California 

Associate Professor of Education, Los Angeles State 
College, Los Angeles, California 

Supervisor of Guidance, Valley- East District, Los 
Angeles City SchoolSi Los Angeles, California 

Research Director, Pasadena City Schools, Pasa« 
dena, California 



Edna Welch 

Nancy A. Westropp 

Mildred WUbar 
Hazel Wilkinson 

James S. Wilkinson 

Margaret C. Williams 
Claude £. Wilson 

John A. R. Wilson 
Uah M. WUstach 
Paul Ci Wilt 
Harrison L. Wiltse 
Arthur C. Wiscombe 

Calvin E. Wright 

Wallace Wright 

WilUamH« Wright, Jr« 

Jerry W^Hk 
Maurice R« Young 

Wayne Zimmerman 
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Addressograph Operatori Mailing List Clerki Gener- 
al ServiceSi Educational Testing Servicei Los AngeleSi 
California 

Psychometristi California State Polytechnic College, 
San Lxiis ObispOi California 

Vice Principali Santa Monica High Schooli Santa Mon*- 
ica^ California 

Elementary Guidance Supervisori East Districti Los 
Angeles City SchoolSi Los Angelesi California 

Giddance Consultanti San Luis Obispo Cotmty School S| 
San Luis ObispOi California 

Psychologist, Little Lake City School Districti Santa 
Fe SpringSi California 

. Consultanti Research and Guidancei Los Angeles 
County Schools, Los AngeleSi California 

Director of Siudeni Teachingi University of CaUfornia, 
Santa Barbara, Universityi California 

Consultanti Research and Guidancei Los Angeles 
Comity Schools, Los Angeles, California 

Acting Dean of Faculty, Westmont CoUegei Santa Bar- 
bara, California 

Psychologist, Arcadia Unified School District, Arcadia, 
California 

Secondary Education Consultant, Los Angeles County 
Schools, Los Angeles, California 

Director, Bureau of Testing, University of Washing-- 
ten, Seattle, Washington 

Intern, Pasadena Counseling Center, Pasadena^ Cal- 
ifornia, 

Guidance Coordinator, Rowland Union School District, 
La Paente, California 

Dean of Men, University of Nevada, Reno, Nevada 

Consultant in Guidance and Research^ Napa County 
Schools, Napa, California 

Test Officer, Los Angeles :tate College, Los Ange- 
les, California 



